SCHOOL TEACHER STUDENTS

ANDRES BONIFACIO COLLEGE CHRISTIE ANGEL ALENSUB BATERNA,

CANDELASA.
CORTEZ
FLORES,
LETEGIO,
LUMAPAY,
NIEZ,

TRUE EXPERIMENTAL DESIGN

• SOLOMON FOUR GROUP CONTROL DESIGN,
• PRE TEST POST-TEST CONTROL,
• POST-TEST ONLY CONTROL GROUP DESIGN GROUP DESIGN
Table of
Contents
Page

I TYPES OF RESEARCH DESIGN 3-4

II TRUE EXPERIMENTAL DESIGN 5-7

III SOLOMON FOUR GROUP CONTROL DESIGN 8-18

IV PRE TEST POST-TEST CONTROL GROUP DESIGN 19-25

V POST-TEST ONLY CONTROL GROUP DESIGN 26-34

I TYPES OF RESEARCH DESIGN
• A method design that allows the researcher to test his
hypothesis by drawing reliable conclusions regarding the
links between independent and dependent variables.
• Theoretical framework within which the experiment is
carried out.
• The primary goal of experimental research is to develop
research with strong causal (internal) validity.
• The highest levels of causal validity are offered by
randomized experimental designs.

3
 There are three main types of experimental research design. As follows:
I

• Pre-Experimental
• True-Experimental
• Quasi-Experimental

4
II TRUE EXPERIMENTAL DESIGN
• The most accurate form of experimental research design as it
relies on the statistical hypothesis to prove or disprove the
hypothesis.
• This is the most commonly used method implemented in
Physical Science.
• True experimental research design is the only method that
establishes the cause-and-effect relationship within the groups.

5
II TRUE EXPERIMENTAL DESIGN

The factors which need to be satisfied in this method are:

• Random variable,
• Variable can be manipulated by the researcher,
• Control Groups (A group of participants is familiar with the
experimental group, but the experimental rules do not apply to them),
• Experimental Group (Research participants where experimental rules are
applied),
• It may be performed on at least two randomly assigned dependent
subjects with or without a pretest. A true experimental research design
must have a control group, avariable that can be modified by the
researcher, and a random distribution.

6

 There are three main types of True Experimental Design. As follows:
II

• Solomon Four-Group Control Design

• Pre test Post-Test Control Group Design
• Post-Test Only/Control Group Design

7
II Solomon Four-Group Control Design
I
• A type of experiment where participants get randomly
assigned to either 1 of 4 groups that differ in whether the
participants receive the treatment or not, and whether the
outcome of interest is measured once or twice in each
group.

8
III Solomon Four-Group Control Design
The objective is to assess the efficacy of the treatment (or intervention).

The four groups in this design are (see figure in the left side):
1. A treatment group with both pre-intervention and post-
intervention measurements (a.k.a. pretest and post-test).
2. A control group with both pretest and posttest
measurements.
3. A treatment group with only a post-test measurement.
4. A control group with only a post-test measurement.

9
III Solomon Four-Group Control Design
The objective is to assess the efficacy of the treatment (or intervention).

The four groups in this design are (see figure in the left side):
1. A treatment group with both pre-intervention and post-
intervention measurements (a.k.a. pretest and post-test).
2. A control group with both pretest and posttest
measurements.
3. A treatment group with only a post-test measurement.
4. A control group with only a post-test measurement.

10
III Advantages & Disadvantages of Solomon Four-Group Control Design

How the Solomon four-group design resembles a two-

group experimental design?

Notice that the first 2 groups of this design resemble the

pretest-post-test control group design. And therefore, the
first 2 groups are enough to control threats to internal
validity such as the effects of:
• Selection
• History
• Maturation
• Mortality
• Regression to the mean
• Instrumentation
11
III Advantages & Disadvantages of Solomon Four-Group Control Design

What makes the Solomon four-group design better than a

standard two-group design?

The Solomon four-group design was developed to:

1. Control threats to internal validity: Such as bias and
confounding. Something that a standard (two-group)
experimental design can control.
2. Control threats to external validity: Such as pretest
sensitization. Something that a standard (two-group)
experimental design cannot control.

12
II Advantages & Disadvantages of Solomon Four-Group Control Design
I
What is pretest sensitization?
• Pretest sensitization (a.k.a. interaction between pretest and treatment) occurs when the use of a pretest increases or decreases
the responsiveness of the participants to the study intervention.
For example:
• Consider a pretest that contains questions that non-deliberately make participants more aware and concerned regarding the
consequences of smoking. Such pretest may sensitize participants to make them more responsive to a smoking cessation
intervention. in fact, it could be that this intervention is only effective for participants who took such pretest.

Why pretest sensitization is an external validity problem?

• The general population from which participants were selected for the study will not be pretested, therefore, will not be
sensitized to the intervention.
• Consequently, the effect of the intervention on the general population will be different than the one detected in the study.
Therefore, the study will suffer from a generalizability (a.k.a. an external validity) problem. 13
II Advantages & Disadvantages of Solomon Four-Group Control Design
I How the Solomon four-group design helps detecting pretest sensitization?
• An interaction between pretest and treatment (or pretest sensitization) would exist if the pretest influences the outcome of the
posttest, only in the presence of the intervention.

Limitations of the Solomon four-group design

• The Solomon four-group design is not used a lot in practice because of its high cost, since compared to a standard design with
2 groups it requires twice the sample size, time, materials, resources, and personnel/work.

14
II Process
I

This can be assessed in 3 steps:

• Step #1: Calculate the effect of the pretest on the

outcome of the posttest in the presence of the
intervention: This is the difference between 1B and 3B.

15
II Process
I • Step #2: Calculate the effect of the pretest on the
outcome of the posttest in the absence of the
intervention: This the difference between 2B and
4B.

• Step #3: Compare the result of step 1 with that of

step 2 (that is “the difference between 1B and 3B”
with “the difference between 2B and 4B”): A
difference between these 2 quantities means that
the effect of the pretest is different in the presence
of the intervention, in other words, there is an
interaction between pretest and treatment.

16
III Using Solomon four-group control design
• It is sometimes used in social science, psychology and medicine. It can be used if there are concerns that the treatment might be
sensitized by the pre-test.

Example of a study that used the Solomon four-group design:

Dehghan et al. investigated VR (Virtual Reality) technology as a method to reduce anxiety in children undergoing surgery.

Design details
• A Solomon four-group design was used in order to control any pretest sensitization.
• The pretest and posttest measured the anxiety score by using a standardized questionnaire.
• The intervention used was a VR technique that simulates step-by-step going into an operation room.
Results
• The study concluded that VR technology reduced pre-operative anxiety in children by acting as a distraction method.
Limitations
• While the treatment allocation was done at random, participants included in the study were sampled in a non-random way from
the population of all children undergoing surgery. 17
III Using Solomon four-group control design
• This means that the study participants may not represent well the population of interest, therefore causing a problem of
generalizability of the study results.(Note : however, that this external validity issue is not specific to the Solomon four-
group design, instead it is a consequence of the sampling method used.)

18
I Pretest-Posttest Control Group Design
V
• Also called the pretest-posttest randomized experimental
design, is a type of experiment where participants get
randomly assigned to either receive an intervention (the
treatment group) or not (the control group). The outcome
of interest is measured 2 times, once before the treatment
group gets the intervention — the pretest — and once after
it — the posttest

The objective is to measure the effect of the intervention which

can be:
• A medical treatment
• An education program
• A policy change, etc. 19
IV Pretest-Posttest Control Group Design
The pretest-posttest control group design has 3 major
characteristics:

1. The study participants are randomly assigned to

either the treatment or the control group (this
random assignment can occur either before of after
the pretest).
2. Both groups are exposed to the same conditions
except for the intervention: the treatment group
receives the intervention, whereas the control group
does not.
3. The outcome is measured simultaneously for both
The pretest-posttest control group is the most commonly used design in groups at 2 points in time — the pretest and the
randomized controlled trials. posttest.

20
IV Advantages & Disadvantages of Pretest-Posttest Control Group Design
Advantages of the pretest-posttest control group design
• By using a pretest, a control group, and random assignment, this design
controls all internal threats to validity.
Advantage of having a pretest measurement
• This design is better than the posttest-only control group design because it
adds a pretest.
Adding a pretest:
1. Increases the power of the design to detect an effect.
2. Allows studying the effect of the intervention at different sublevels of the
pretest.
3. Helps analyzing initial differences between groups (and therefore
quantifying their effect on the study outcome).
4. Helps controlling attrition bias i.e. the unequal loss to follow-up of
participants between the treatment and the control group which can affect
the outcome measured at the posttest.
21
I Advantages & Disadvantages of Pretest-Posttest Control Group Design
V
Advantage of using random assignment and having a control group:
Random assignment and the control group will both limit the effects of:

1. Selection bias: Which happens when participants themselves get to choose if they receive the intervention or not. This may
create unequal and incomparable study groups. Randomization allows unbiased assignment of participants to treatment
options, and therefore makes the study groups comparable.
2. Maturation: Which is the effect of time (between the pretest and the posttest) on study participants (e.g. participants
growing older, or getting tired over time) which might influence the outcome, thus becoming a rival explanation for the
intervention regarding the study outcome. Participants are subject to maturation both in the treatment and the control
group, therefore, any difference between the outcome of these groups will be due to the effect of the treatment alone and
will not be affected by maturation.

22
IV Advantages & Disadvantages of Pretest-Posttest Control Group Design
3. History: Which is any event that might co-occur with the intervention and has the potential to influence the outcome.
Co-occurring events affect both the treatment and the control group, and therefore any difference between the outcome of
these groups will be due to the effect of the treatment alone and will not be affected by history.

4. Testing: Which is the effect of taking a pretest on the result of a posttest. For instance, if the pretest sensitizes
participants and compels them to behave in a certain way that affects the outcome of the posttest. The presence of a
control group protects against testing effects, as these will affect both groups and therefore any difference between the
outcome of these groups will be due to the effect of the treatment alone and will not be affected by testing.

5.Regression to the mean: When pretest scores are exceptionally good by chance, the posttestscores will naturally regress
toward the mean. This happens because an exceptionally good performance is hard to maintain. Regression toward the
mean can be mistaken for the effect of the treatment, and therefore is a source of bias. Since participants from both
groups are subject to regression, therefore, comparing the outcome of the treatment group with that of the control group
will take care of this regression effect.

23
IV Advantages & Disadvantages of Pretest-Posttest Control Group Design

Limitations of the pretest-posttest control group design

Participants included in any randomized study might not be typical people in the population i.e. they may not represent well
the population of interest, this is because:
1. Not everyone in the population of interest is eligible for the experiment,
2. and not everyone who is eligible can be recruited,
3. and not everyone who is recruited will give us their consent to be included in the study,
4. and not everyone who consented will be randomized.

So the outcome of a randomized study may not generalize well to the population.
More specifically, this design:
• Does not allow us to study how the effect of the treatment changes over time: To do so, we need to add more posttest
measures.
• Is susceptible for interactions between the intervention and other factors (such as the pretest, history, instrumentation, etc.):
One solution for this problem is to use the Solomon four-group design.
24
IV Using Pretest-Posttest Control Group Design
A pretest needs to be administered first on the participants. This is followed by the treatment. A post-test will be conducted after the
treatment. An analysis of pretest and posttest data needs to be completed by measuring the dependent variable before and after
exposure to the treatment.

Example of a study that used the pretest-posttest control group design:

Koenig et al. used a pretest-posttest control group design to study the effect of a yoga program on the classroom behavior of autistic
children. These children were randomly assigned to either receive the yoga program or their standard morning routine.

The study concluded that yoga can significantly improve the classroom behavior of autistic children.
But because the researchers used a convenience sample from a particular school and the classrooms that were allowed to participate
were hand-picked by administrators, the study outcome may not generalize well to all children with autism.

25
V Post-test Only Control Group Design

• A basic experimental design where participants get

randomly assigned to either receive an intervention or not,
and then the outcome of interest is measured only once
after the intervention takes place in order to determine its
effect.

26
V Post-test Only Control Group Design

The intervention can be:

• a medical treatment
• a training program
• an exposure to a risk factor, etc.

Note that this design differs from the pretest-posttest randomized controlled trial by having no measurements taken before the
intervention. Also, if we remove the random assignment component from this design (and let participants get assigned to groups
according to their choosing or that of the researcher), we get the static-group comparison design which is a type of quasi-
experiment.

27
V Advantages & Disadvantages of Post-test Only Control Group Design

Advantages of the posttest-only control group design:

1. The treatment and control groups are equivalent at baseline

• This is very important because when the 2 groups are equal, any difference in the outcome measured will be
attributed only to the intervention and not to the initial difference between the groups.
• Note that these groups are not expected to be perfectly equal, in fact they don’t need to be. As long as we are
assigning participants at random, we won’t be subject to selection bias and we will obtain comparable groups.
• This is an advantage over static-group comparison where assignment is not done at random, so the initial
difference in the characteristics of both groups can be an important factor influencing the outcome of the study
and therefore an important source of bias.

28
V Advantages & Disadvantages of Post-test Only Control Group Design
2. External factors are controlled
• The use of a control group controls history (i.e. adjusts for the effects of events that can happen at the same time as
the intervention and can influence the outcome, therefore becoming a rival hypothesis and a potential source of bias).
• The use of 1 measurement only controls factors related to the instruments used to measure the outcome (since the
device is only used once to measure the outcome, we won’t have to deal with changes that can happen to the device
or to the quality of measurements from 1 measurement to the next).
• The simultaneity in measuring both groups controls factors that change with time (as these may also affect the
outcome and bias the study).

3. Can be used when participants’ anonymity must be kept

• When subjects are measured before and after the intervention, some sort of system should be installed in order to
know which measurement corresponds to which participant. In these designs, the participant’s ID, name or phone
number will be recorded in a database which may interfere with some type of studies where the participant’s
anonymity is critical.

29
V Advantages & Disadvantages of Post-test Only Control Group Design
4. Not affected by reactions to pretesting
• An additional source of bias may be present when posttest results can be influenced by the results of a pretest. For
instance:
-a physician might be influenced by a previous diagnosis or opinion made on a patient
-a participant taking the test the second time may be more prepared to the type of questions asked, etc.

So a study involving a pretest may be measuring the effect of the intervention along with this test-retest effect (a.k.a.
sensitization to pretest). As this design does not have any pretest, it will not be subject to this bias.

5. Can be done when a pretest is not possible

• This is especially useful when the measurement itself is very expensive either in terms of time or money, or when its
results can be easily predicted or constant. For instance, a pretest is unnecessary if the outcome we want to measure
is mortality and all participants are alive at the start of the study.

30
V Advantages & Disadvantages of Post-test Only Control Group Design
Limitations of the posttest-only control group design

1. High risk of attrition bias

• This is due to participants quitting the study for different reasons between the study groups making the groups
unequal anymore. Note that the risk of attrition bias is higher when the intervention takes a long period of time to be
implemented (in the order of days or weeks).
• The absence of a pretest makes it very hard to detect and control this bias.
2. The effect of the intervention on subgroups is not clear
• In this posttest-only design we cannot compare the outcome with pretest measures, meaning that we cannot
investigate which subgroup of participants responded more to the treatment or which subgroup did not respond well.
3. Requires a large sample size
• The minimum sample size required for this study will be larger compared to studies without random assignment (i.e.
quasi-experiments), this is because randomness can correct the differences between the treatment and control groups
better as the sample gets larger.

31
V Advantages & Disadvantages of Post-test Only Control Group Design
4. Less generalizable than observational designs
Low external validity is a general characteristic of experiments for 2 reasons:
• participants who agreed to be part of the experiment may be different from those in the population on
which we would like our results to generalize.
• experiments happen in a closed and controlled environment which is not always representative of a real
world scenario.

Observational studies however, do not have such limitations as the investigator is just an observer of natural
events, watching and recording them as they happen without controlling or influencing them. In general, a
highly controlled study will have a high internal validity (i.e. less bias) and a low external validity (i.e. low
generalizability).

32
V Using of Post-test Only Control Group Design
• The posttest-only control group design is almost the same as classic experimental design, except it does not use a
pretest. Researchers who use posttest-only designs want to eliminate testing effects, in which participants' scores
on a measure change because they have already been exposed to it.

Example of a study that used a posttest-only control group design:

In 1993, Topf and Davis used a posttest-only control group design to examine if CCU (Critical Care Unit) noise
affects REM (Rapid Eye Movement) sleep.
So they randomly assigned 70 women with no hearing or sleeping problems to attempt to sleep in one of the
following conditions:
• noisy environment (the subjects listened to an audiotape recording of CCU sounds): treatment group
• quiet environment: control group

Note that this experiment was done in a sleep laboratory.

Their results showed that CCU sounds can cause poorer REM sleep.

33
V Using of Post-test Only Control Group Design

What can we learn from this example?

1. The absence of a pretest was justified because participants had no sleeping problems before the experiment.
2. This study may suffer from a generalizability issue because it used laboratory subjects that may not be
representative of the entire population.

34