6

Introduction to Design

Foundations of Design
Research design is the glue that holds
pieces of the research project together
The sample
The measures
The treatments or programs
The method of assignment

Research Design and Causality

Causal
Pertaining to a cause-effect question,
hypothesis, or relationship
Something is causal if it leads to an outcome
or makes an outcome happen
(Not casual)

Top Threat: The Third-Variable Problem

An unobserved variable that accounts for
a correlation between two variables
Correlation does not equal causation!
Minimal criteria for causality:
Temporal precedence
Covariation of the cause and effect
No plausible alternative explanations

Hills Criteria for Causality

Temporal precedence
Strength of association
Dose-response relationship
Consistently generalizable elsewhere
Plausibility
Consideration of competing hypotheses
Mutable by experimentation
Coherence with existing frameworks

Internal Validity

Internal Validity
The three main types of threats
Single-group threats
Multi-group threats
Social-interaction threats

Internal Validity Single Group Threats

Occurs in single group post-test or
pretest-post-test designs
Types:
History
Maturation
Testing
Instrumentation
Mortality (Attrition)
Regression to the mean

Internal Validity Single Group Threats

(contd.)
Deal with single-group threats by adding a
second group!
Called a control group in an experiment,
comparison group in quasi-experiment

Once youve added a control group, you

need to address the multi-group threats to
internal validity

Internal Validity Multi-Group Threats

There really is only one multiple-group
threat to internal validity: that the groups
were not comparable before the study:
Selection bias

Internal Validity Types of Selection Bias

Selection-history threat
Selection-maturation threat
Selection-testing threat
Selection-instrumentation threat
Selection-mortality threat
Selection-regression threat

Internal Validity Multi-Group Threats

Randomization to treatment groups is
the defining characteristic of an
experiment

Types of Designs

Random Assignment vs. Random

Selection

Internal Validity Social-Interaction

Threats

Diffusion or imitation of treatment

Compensatory rivalry
Resentful demoralization
Compensatory equalization of treatment

Developing a Research Design

Four elements to any research design:
Time
Treatments or programs (IV/predictors)
Measures or observations (DV/outcomes)
Groups or individuals

Design Notation

Notational Examples

Expanding on Basic Designs

Expanding across time

Expanding across programs
Expanding across observations
Expanding across groups
The classic pretest-posttest control group
design

Different designs have different

weaknesses
Post-test only, two-group,
randomized experiment

Switched-replication,
two group experiment

More designs
Time-series experiment

Pre-and-post-test, two-group,
non-equivalent groups quasiexperiment

Discuss and Debate

Why is internal validity so important in
research design?
What is the purpose of randomization in
research design?