Experimental, Quasi-

Experimental, and Ex Post Facto

(Causal-Comparative) Research
Characteristics of
Experimental Research
There is a control or comparison group
Subjects are randomly assigned to
groups
The treatment is randomly assigned to
groups.
Characteristics of Quasi-
Experimental Research
There is a control or comparison group
Intact groups are used
The treatment is randomly assigned to
groups.
Characteristics of Ex Post
Facto Research
There is a control or comparison group
Intact groups are used
The treatment is not manipulated, it
has already occurred.
Diagramming Research
To illustrate research designs, a number
of symbols are used
X
1
= Treatment
X
2
= Control Group
O = Observation (pretest or posttest)
R = Random Assignment
A Sample Research Design
Single-Group Pretest-Treatment-
Posttest Design
R O X
1
O
This means subjects are
randomly assigned to a
group, which is then given a
pretest, then there is a
treatment, then there is a
posttest.
R O X
1
O

This is not really an experimental design
because there is no control group
It is often referred to as a preexperimental design
Novice researchers often use this research
design
There are some major problems with this
design did the treatment really make the
difference or was something else happening.
R O X
1
O

What are the threats to the Internal
Validity of this type of research (Did
the treatment really cause a
difference?)
Internal Validity Threats
History
Another event occurs during the time of the
experiment that might cause the difference
An experiment to heighten racial awareness was
conducted by a researcher during February. This is
Black History month; so the results might be affected
by events that occur during Black History month and
not the treatment.
R O X
1

O
Internal Validity Threats
Maturation
People naturally change and evolve over time.
This may cause the difference.
A college develops a new housing plan to promote
more open-mindness and acceptance of others. The
students are tested when they enter college and when
they graduate. The results show they are now more
open-minded and tolerant of others. Did the housing
plan work or do students just mature and grow as a
result of the college experience.
R O X
1

O
Internal Validity Threats
Mortality
Some people drop out during an experiment. This
may affect the outcome.
I am teaching a new experimental seminar on study
skills. About half of the class stopped coming to the
seminar before the semester was over. The students
who remained improved their study skills. So my
course was effective!
Probably not. The half that stopped coming might not
have gained anything; that is why they stopped
attending.

R O X
1

O
Internal Validity Threats
Testing
Whenever you give a pretest, the students may
remember the test questions, and get them correct
on the posttest.
I gave a test to my study skills group on Monday,
presented some unique concepts on Tuesday, then gave
them the posttest on Wednesday. The grades were
significantly higher on the posttest.
It is possible the grades were higher because the students
still remembered the questions from the pretest.
R O X
1

O
Internal Validity Threats
Instrumentation
To overcome the testing threat to internal
validity, a researcher develops a different form of
the test instrument, but it is not really equivalent.
I gave a test to my study skills group on Monday,
presented some unique concepts on Tuesday, then gave
them an alternative form of the pretest on Wednesday.
The grades were significantly higher on the posttest.
It is possible the grades were higher because the second
test was easier than the first.
R O X
1

O
Internal Validity Threats
Regression
When subjects are selected because of extreme
scores on some type of instrument, there is
tendency for their scores to move more toward
the average on subsequent tests.
An experimenter selected students for a reading
program based on their low test scores. At the end of
the treatment, the test scores had improved.
Extreme scores naturally move toward the mean on
subsequent tests.
O X
1
O
How to Handle Internal
Validity Threats
Have a control group and use
randomization.This design is the Two-
Group Pretest-Treatment-Posttest
Design.
R O X
1
O
R O X
2
O
The Control Group would experience the same
history and maturation. Mortality should be the
same because of random assignment. Random
assignment eliminates the selection threat.
However testing and instrumentation could still
be a threat.
Other Research Designs
Two-Group Treatment-Posttest-Only
Design
R X
1
O
R X
2
O
There is no pretest so this
eliminates the testing and
instrumentation threat to
internal validly but you
dont know about their
knowledge or attitude
coming into the study.
Other Research Designs
Solomon 4-Group Design
R O X
1
O
R X
1
O
R O O
R O
Note: A blank
indicates the
control group,
same as X
2
Quasi-Experimental Designs
Posttest Only Nonequivalent Group
Design
X
1
O
X
2
O
The absence of R indicates
there is no random
assignment. Sometimes you
will see a dotted line
between the two groups.
This indicates the two
groups may not be
equivalent.
Quasi-Experimental Designs
Pretest-Posttest Nonequivalent Group
Design
O X
1
O
O X
2
O
Time Series Designs
O O O X
1
O O O

In the next course, AEE 579 Research Design,
many more research designs are examined.
External Validity
Can the research be generalized to
other settings?
Population Validity
Personological Variables
Ecological Validity
Population Validity
Is the sample population similar to the
population the researchers wishes to
generalize to
Personological Variables
Different people have different
personalities, learning styles, etc., so
the results may not be generalizable to
people who are substantially different
on these personological variables.
Ecological Validity
The setting or situation in which the
experiment occurred may be different
than other settings.
Social Interaction Validity
Threats
Diffusion or Imitation of Treatment
This occurs when a comparison group learns
about the program either directly or indirectly
from program group participants.
This group may try to imitate or emulate what the
treatment group is getting.

Social Interaction Validity
Threats
Compensatory Rivalry
The comparison group knows what
the program group is getting and
develops a competitive attitude with
them.
Social Interaction Validity
Threats
Resentful Demoralization
This is almost the opposite of compensatory
rivalry. Here, students in the comparison
group know what the program group is
getting. But here, instead of developing a
rivalry, they get discouraged or angry and
they give up.

Social Interaction Validity
Threats
Compensatory Equalization of
Treatment
The researcher is under pressure to
enrich the experiences of the control
group. This pressure may come from
parents, school administrators, etc.
Ex Post Facto (Causal-
Comparative) Research
Explores possible causes and effects
The independent variable is not manipulated,
it has already been applied
Focuses first on the effect, then attempts to
determine what caused the observed effect.
Statistical Analysis
If we are comparing the scores of two
groups a t-test is normally used. The
value of t means nothing by itself
(unlike the value of R). We have to
determine if t is statistically significant
Tea for two
Statistical Analysis
If we are comparing the scores of three
(or more) groups Analysis of
Variance (ANVOA) is used. This test
gives us a f value which means nothing
by itself. We have to determine if it is
statistically significant.
Statistical Analysis
If we want to statistically equate two or more
groups (because one group had a high pretest
score) we use Analysis of Covariance
(ANCOVA). This test gives us a f value
which means nothing by itself. We have to
determine if it is statistically significant.