SUB-MODULE 4:

EXPERIMENTAL
STUDIES VS NON-
EXPERIMENTAL
DESIGNS

1
 4. EXPERIMENTAL VS. NON- EXPERIMENTAL STUDIES

LEARNING OBJECTIVES

By the end of the session, the learner should be able to:

1. Differentiate between descriptive and analytic study designs;

2. Identify the different descriptive and analytic study designs used in epidemiologic
studies;
3. Describe the advantages and disadvantages of each epidemiologic study design;
4. Describe the main characteristics of experiments

4.1 PREVIEW

This sub-module will present the different observational and experimental study designs used in
health research. A description of the different descriptive and analytic epidemiologic study
designs are presented, together with their advantages and disadvantages. The main attributes
of experimental studies are also discussed.

4.2 TYPES OF STUDY DESIGNS

There are many ways of categorizing study designs, depending on which dimension you
are looking at. A very common way of categorizing them is whether the study design is
descriptive or analytic, depending on the research objectives.

Descriptive studies are generally exploratory in nature, and hence the main question
being answered is “What”? Analytic studies on the other hand are explanatory and
hence aim to answer the question “Why”? The results of descriptive studies are used as
basis for formulating hypothesis, which are tested in analytic studies. A detailed
comparison between descriptive and analytic studies as presented by Aday (1986) in
shown in Table 1.

2
 Table 1. Comparison of descriptive and analytic studies

DESCRIPTIVE ANALYTICAL

Describes Explains

Is more exploratory Is more explanatory

Profiles characteristics of Analyzes why group

group has characteristics

Focuses on “what” Focuses on “why”

Assumes no hypothesis Assumes a hypothesis

Does not require Requires comparisons

comparisons between between groups over
groups or over time time

In health, most researches are in the form of epidemiologic studies. There are several kinds of
epidemiologic study designs which can likewise be categorized into descriptive and analytic
studies. The different types of epidemiologic study designs are presented in Figure 1.

3
 DESCRIPTIVE ANALYTIC
Describe a disease or health Examine association (test of
condition/phenomenon or hypothesis)
intervention

Types Observational Experimental

1. Case reports/Case Exposure and outcome Exposure
series variables are just variables are
2. Prevalence survey observed assigned
(cross-sectional or
cohort)
3. Ecologic study

Types Types
1. Cross-sectional 1. Clinical trials (RCT)
2. Case-control 2. Field trials
3. Cohort 3. Community
4. Ecologic intervention trials

Identification of risk/protective factors

Hypothesis formulation

Figure 1. Types of epidemiologic study designs

4.2.1 Descriptive study designs

As shown in Figure 1, there are three kinds of descriptive study designs: case studies or
case series; descriptive cross-sectional studies/prevalence surveys and ecologic
descriptive studies.

A case study is a simple descriptive account of interesting characteristics observed in a

subject. If there are several subjects, the study design is called a case series. The subjects
do not necessarily have to be persons – they can be organizations, institutions, political
units (province, municipality, barangay etc.). Case studies generally involve subjects seen
over a relatively short period of time. Because a case study is descriptive, it does not
include control subjects. Neither is a hypothesis being tested. The results of case studies
generally lead to generation of hypothesis that are subsequently investigated in a cohort,
case-control, or cross-sectional study. Hence in the hierarchy of study designs based on
validity and statistical power, the case study is at the bottom.

4
Descriptive cross-sectional studies/prevalence surveys involve the collection of data on
the occurrence and distribution of the disease of interest in populations. To characterize the
disease, the prevalence is usually computed for specific categories of variables related to:
 person (sex; age; occupation, etc.);
 place (geographic subdivisions; type of terrain, etc); and
 time (month; season, etc.).

Cross-sectional studies are also used in health systems research to describe prevalence
according to patterns of health service utilization and compliance. KAP surveys may also
be categorized under this category

Ecologic studies differ in that the unit of observation and unit of analysis is an aggregate
rather than individual persons. It is the most practical design to use when exposure level is
relatively homogeneous in a population but differs between populations (ex., water quality)
or when individual measurements of exposure are impossible (ex., air pollution). They are
used to generate hypothesis, or as a quick method of examining associations. Its most
serious flaw is the risk of ecological fallacy -- i.e., the characteristics of the geographical
unit are incorrectly attributed to individuals.

4.2.2 Analytic study designs

There are four types of analytic study designs: cohort, case control, cross-sectional and
experimental studies. The first three are all observational study designs. In the context of
epidemiologic studies, these study designs are used to investigate the relationship between
an exposure factor and an outcome variable, which is usually the incidence or prevalence
of a disease.

Before we describe the structure of each of the observational study designs, it is important
to emphasize the core elements which need to be present when investigating the
relationship between exposure and outcome. This is shown in Table 2.

Table 2. Core elements in the investigation of the relationship

between disease and exposure in observational studies .

DISEASE Exposure Status

STATUS Exposed Unexposed
With the disease
Without the disease

The core elements presented in Table 2 show that in order to investigate the relationship
between disease and exposure, we need four types of subjects represented by the body of
the table: with the disease and exposed; with the disease and unexposed; without the
disease and exposed; without the disease and unexposed. For example, in order to
determine the relationship between smoking and lung cancer, one cannot establish such a
relationship without having four types of study subjects: those without lung cancer and are
smokers; those without lung cancer and are non-smokers; those with lung cancer and are
smokers; and those with lung cancer and are non-smokers. Without these four groups of

5
subjects, appropriate comparisons cannot be made, and the relationship between the
dependent (outcome) and independent (exposure) variables cannot be established.

4.2.2.1 Cohort study

In a cohort study, the sampling population from which subjects are selected are those who
are free of the disease being studied. From this population, a sample of exposed and
unexposed individuals will be selected. They will be followed-up through time to determine
their disease status at end the study. This is shown in Table 3.

Table 3. Set-up of a cohort study

SAMPLING SAMPLES TO BE SELECTED DATA TO BE COLLECTED

POPULATION AT START OF STUDY IN THE STUDY*
Exposed individuals Number with the disease
Population without Number without the disease
the disease of interest Unexposed individuals Number with the disease
Number without the disease
* Data will be collected after a follow-up period, the length of which varies according to the
nature (latent/incubation period) of the disease being studied

As an example, suppose we do a cohort study to determine if physical activity level

is a risk factor for hypertension. The corresponding set-up of this study is shown in
Table 4.

Table 4. Set-up of a cohort study to determine the relationship between

hypertension and physical activity level

SAMPLING SAMPLES TO BE SELECTED DATA TO BE COLLECTED

POPULATION AT START OF STUDY IN THE STUDY
Individuals with sedentary life Number with hypertension
Population without style Number without hypertension
hypertension Individuals with active life style Number with hypertension
Number without hypertension

Cohort studies have the following advantages:

a. It yields information on the incidence of the disease.

b. It is possible to compute for the relative risk
c. The temporal relationship between exposure and disease is clearly defined.
d. The design is particularly efficient for studies involving rare exposure factors.
e. It is the strongest observational design for establishing cause-effect relationships.

6
Its disadvantages are as follows:
a. It is time-consuming.
b. It often requires a large sample size.
c. It is expensive.
d. It is not efficient for the study of rare diseases.
e. Losses to follow-up may diminish validity.
f. Changes over time in diagnostic methods may lead to biased results

There are actually two types of cohort studies: prospective and retrospective cohort. In a
prospective cohort, the exposure factor is assessed at the start of the study while the
outcome is followed up into the future. In contrast, in a retrospective cohort, the exposure is
assessed at some point in the past for which records are available; the outcome on the
other hand, has already occurred. It is a difficult design to implement and is usually used
when there is simultaneous exposure to a factor (ex., natural and man-made disasters).

4.2.2.2 Case-control study

The set-up of a case-control study is presented in Table 5. It is the opposite of the cohort
since its starting point is the disease status instead of the exposure. This is usually done
when takes a long time for the disease to develop, or when there are only a few cases of
the disease, thereby making the conduct of a cohort study impractical. What needs to be
collected in the course of the study is the exposure status, which is determined
retrospectively through personal interviews or records review, if these are available.

Table 5. Set-up of a case-control study

SAMPLING SAMPLES TO BE SELECTED DATA TO BE COLLECTED

POPULATION AT START OF STUDY IN THE STUDY*
1. Population with the Individuals with the disease Number of exposed individuals
disease (cases) (cases) Number of unexposed
individuals
2. Population without the Individuals without the disease Number of exposed individuals
disease (controls) (controls) Number of unexposed
individuals
* Exposure data will be collected retrospectively through personal interviews and/or records review

Table 6 shows the set-up of a case-control study, using the same example of the study on
hypertension and physical activity level.

7
Table 6. Set-up of a case-control study to determine the relationship between
hypertension and physical activity level

SAMPLING SAMPLES TO BE DATA TO BE COLLECTED

POPULATION SELECTED IN THE STUDY*
AT START OF STUDY
1. Population with Individuals with Number of individuals with sedentary lifestyle
hypertension (cases) hypertension (cases) Number of individuals with active lifestyle
2. Population without Individuals without Number of individuals with sedentary lifestyle
hypertension hypertension (controls) Number of individuals with active lifestyle
(controls)

The advantages of a case-control study include the following:

a. It is feasible when dealing with rare diseases

b. It requires a smaller sample size than a cohort study
c. There is little problem with attrition since the study does not require a follow-
up period and hence takes only a short time

Among its disadvantages are as follows:

a. Incidence rates and attributable risks cannot be computed.

b. The temporal sequence between disease and exposure may be a problem
c. There is a big chance for bias in the selection of cases and controls
d. It may be difficult to obtain information on exposure if the recall period is too long.
e. Selective survival may bias the comparison.

There are two kinds of case-control studies: population-based and hospital-based case-
control studies. In population-based case-control studies, cases and controls are sampled
from a defined population like a municipality or a barangay. An advantage of this study
design is that its source population is better defined. It is also easier to make certain that
cases and controls come from the same source population. The exposure histories of the
controls are more likely to reflect those of persons without the disease of interest.

In hospital-based case control studies, the investigator selects cases from persons with the
disease of interest who are admitted to a particular hospital. Controls are selected from
persons admitted with other conditions but with no evidence of the disease of interest. An
obvious advantage of hospital-based case control studies is that subjects are more
accessible and tend to be more cooperative. Background characteristics of cases and
controls may be balanced. It is also easier to collect exposure information from medical
records and biologic experiments

4.2.2.3 Cross-sectional study

For both cohort and case-control studies, the investigator needs to already have an idea of
either the subjects’ disease or exposure status, at the start of the study. However, there are
many instances when such data are not available. This is when the conduct of cross-
sectional studies may be more appropriate.

8
The set-up of a cross-sectional study is shown in Table 7. It differs from the two other study
designs in that both the exposure and outcome are simultaneously determined at the time
of data collection. The set-up for the example on hypertension and physical activity level
using a cross-sectional study design is presented in Table 8.

Table 7. Set-up of a cross-sectional study

SAMPLING SAMPLES TO BE SELECTED DATA TO BE COLLECTED

POPULATION
Population with both disease
and exposure status
unknown at start of study
AT START OF STUDY
Sample of individuals from the
target population with both
disease and exposure status
unknown at start of study
IN THE STUDY
Number of exposed individuals
with the disease
Number of exposed individuals
without the disease
Number of unexposed
individuals with the disease
Number of unexposed
individuals without the disease

Table 8. Set-up of a cross-sectional study to determine the relationship

between hypertension and physical activity level

SAMPLING SAMPLES TO BE SELECTED DATA TO BE COLLECTED

POPULATION
Population whose physical
activity levels and status
with respect to hypertension
are unknown at start of
study
AT START OF STUDY IN THE STUDY
Number of hypertensive
Sample of individuals from the individuals with sedentary
target population whose lifestyle
physical activity levels and Number of normotensive
status with respect to individuals with sedentary
hypertension are unknown at lifestyle
start of study Number of hypertensive
individuals with active lifestyle
Number of normotensive
individuals with active lifestyle

The advantages of a cross-sectional study are as follows:

a. It is less time-consuming and less costly than prospective studies

b. They often serve as the starting-point in prospective cohort studies for screening-
out already existing conditions.
c. The design allows the measurement of risk, although the estimate is not precise.

9
Among its disadvantages are as follows:

a. It does not enable the direct estimation of risk.

b. It is prone to bias from selective survival.
c. It is often difficult to establish the temporal sequence of exposure factor and the
disease

4.2.2.4 Comparisons of study designs

A comparison of the attributes of the three observational analytic study designs presented
earlier is shown in Table 9.

Table 9. Comparison of cohort, case-control and cross-

sectional study designs on selected attributes

ATTRIBUTE COHORT CASE-CONTROL CROSS-SECTIONAL

Sampling population Population without the 1. Population with the Population with both
disease disease (cases) disease and exposure
2. Population without the status unknown at start
disease (controls) of study
Temporal sequence  Prospective (for Retrospective Current and/or
prospective cohort) retrospective
 Retrospective
(for
Retrospective
cohort)
Use Compares incidence Compares prevalence of Describes association
rates in exposed and exposure among cases between exposure and
unexposed and controls disease
Measure of disease Incidence of disease Cannot be computed Prevalence of disease
frequency among exposed and among exposed and
unexposed groups exposed
Measure of association • Relative risk Odds ratio (estimate of • Prevalence ratio
between disease and • Attributable risk relative risk) (inexact estimate of
exposure relative risk)
• Odds ratio

4.2.3 Experiments

Experiments provide the best evidence for testing any hypothesis or to investigate possible
cause-effect relationships. They resemble cohort studies in that they require follow-up of
subjects to determine outcome. Its essential distinguishing feature is that it involves action,
manipulation or intervention on the part of the investigator.

10
The main characteristics of an experiment are as follows:
a. Pre and post-treatment measurements are made.
b. Presence of a control group
c. Subjects are randomly selected from a reference population
d. Subjects and treatments are randomly assigned to groups
e. High degree of control over extraneous variables

Experimental designs are called differently depending on the discipline. For example,
agriculturists talk about completely randomized and randomized complete blocks designs,
while clinicians use the term randomized controlled trials. However it is called, the basic
set-up of an experiment is shown in Figure 2.

Pre-test Treatment Post-test

Experimental Group1
T 1 X T 2

Control Group1
T1 - T2

1 Random allocation of subjects

Figure 2. Basic set-up of an experiment

The reason why a full experiment requires pre and post-treatment measurements is to
enable the measurement of change resulting from the treatment. Change is often used as
indicator of effectiveness.

The presence of a control group on the other hand is needed to determine whether or not
change occurs even in the absence of the treatment or intervention. To illustrate this, let us
suppose that a health education program on breastfeeding was conducted in 3
municipalities. As an indicator of effectiveness, the percentage of mothers breastfeeding
their babies was determined before and after the program. Suppose there was no control
group in the study, and the results are shown in Figure 3. Since all the percentages
increased before and after in the 3 municipalities, then one might conclude that the
intervention is effective in all areas.

Figure 4 presents another scenario for the same study, but with a control group. The
figures presented graphically in terms of a broken line In Figure 4 are the same as those in
Figure 3. With the presence of a control group, it can be seen that now the conclusions are
different. For Municipality A, the intervention is not necessary because an increase in the
number of mothers breastfeeding their babies is observed even in the control group. For
Municipality C, the situation is even worst because the increase in the percentage of
mothers breastfeeding their babies is higher in the control group compared to the study
group. The effectiveness of the intervention is demonstrated only in Municipality B where

11
there was a big increase in the percentage of mothers breastfeeding their babies in the
study group, but no changes were observed in the absence of the intervention.

FIGURE 3. PERCENTAGE OF MOTHERS BREASFEEDING THEIR BABIES BEFORE AND

AFTER A HEALTH EDUCATION PROGRAM: MUNICIPALITIES A, B AND C

Percentage of Mothers Breastfeeding

Percentage of Mothers Breastfeeding Percentage of Mothers Breastfeeding Their
Their Babies: Municipality A
Their Babies: Municipality B Babies: Municipality C

90 80 70
80 70 60
70
60
50
60
50
50 40
Percentage

40

P
40

c
r
30
30 Percentage 30
20 20
20

10 10 10

0 0 0
Before After Before After Before After

Effective Effective Effective

12
 FIGURE 4. PERCENTAGE OF MOTHERS BREASFEEDING THEIR BABIES BEFORE AND
AFTER A HEALTH EDUCATION PROGRAM: MUNICIPALITIES A, B AND C

Percentage of Mothers Breastfeeding Percentage of Mothers Breastfeeding Percentage of Mothers Breastfeeding

Their Babies: Municipality A Their Babies: Municipality B Their Babies: Municipality C

90 80 80
80 70 70
70 60
60
60
50 50
With Health Educ
50 With Health With Health
Educ. 40 Educ. 40
Without Health
Percentage

40 Without Health Without Health

Educ. Educ.
Educ

n
P
e

e
a
g
c
r

t
30
n
P
e

e
a
g

30
c
r

30

20 20
20
10
10 10
0
0 0
Before After
Before After Before After

Effective but not necessary Effective Not effective

When subjects cannot be assigned at random to the different treatment groups, that is what
we call a quasi-experimental design.

Figures 5 and 6 show variations in the set-up of experimental designs. Figure 5 shows a
design wherein several post-treatment measurements are made. This design is usually
used when it is the sustained or the long-term effect of the intervention which is of interest
in the study, and not the immediate effect. Figure 6 on the other hand shows a design
where the effect of several treatments or interventions are studied.

13
 Figure 5. Experimental set-up with several post-treatment measurements

Pre-test Treatment Post-test

Experimental Group 1
T X
T
1 2

Experimental Group 2
T X
T
1 2

………….. T1 X T2
Experimental Group t
T X
T
1 2

Control Group T1 - T2

Figure 6. Experimental set-up with several treatment groups

4.2.3.1 Experiments in drug development

One area where a lot of experimentation is done is in drug development. It is a very long and
tedious process which starts with pre-clinical studies for purpose of isolating and characterizing
active compounds of interest, and testing of its absorption, distribution, metabolism, excretion
and toxicological properties (ADME/Tox). This is followed by pharmacology and toxicology
studies in animals with the goal of establishing that there are no observable adverse effects.
Levels. The dosage to be used for initial Phase 1 clinical trial of the drug also need to be
determined.

The last stage in drug development is the conduct of clinical trials. These are experiments
conducted by clinicians and epidemiologists to evaluate drugs, medical devices and clinical or
health care procedures. The most common form of a clinical trial is the randomized, controlled,
double blind clinical trial.

The conduct of clinical trials has 4 phases as shown in Table 10. The large clinical trials usually
done by doctors in the hospitals are part of Phase 3 clinical trials.
 Table 10. Phases of a clinical trial

PHASE 1 • Perform initial human testing in a small group of healthy volunteers (about 20-
100)
• Major goal is to determine if drug is safe in humans
PHASE 2 • Test in a small group of patients (about 100 – 500)
• Objective is to determine possible short-term side effects and risks associated
with the drug; if it works according to expected mechanism
PHASE 3 • Test in a large group of patients (about 1000-5000) to show safety and efficacy

PHASE 4 • Post-marketing surveillance of drug to determine long-term safety and

reassess effectiveness, acceptability and continued use under normal field
settings

REFERENCES

Aday, L.A. (1996). Designing and Conducting Health Surveys A Comprehensive Guide (Second
Edition). Jossey-Bass Publishers. San Francisco.

Bonita, R. Beaglehole, R. and Kjellstrom, T. ( 2006). Basic Epidemiology. World Health

Organization. Geneva.

Mausner, J. and Bahn, A. (1974). Epidemiology An Introductory Text. W.B.Saunders Company.

Philadelphia, PA.

15