Experiment Design 5: Variables & Levels

Martin, Ch 8, 9,10

Recap

Different kinds of variables

Independent, dependent, confounding, control, and random

Different kinds of validity

Internal, construct, statistical, external Each associated with a question

Randomization
Random sampling: generalization Random assignment: causation

Picking a design

Choosing how to assign participants to levels of an independent variable

Between vs. within

Choosing how many levels of an independent variable Choosing how many independent variables

Between vs. Within designs

Condition 1:
Fred Ginger Mary 5 8 6

Condition 1:
Fred Ginger Mary 5 8 6 6 9 7

Condition 2:
Ed Mabel George 6 9 7

Condition 2:
Fred Ginger Mary

Within vs. Between Subjects

Cost
Between: More participants Within: More time per participant

Confounding variables
Between: Group differences possible
Use randomization, many subjects, matching

Within: Order effects possible

Use counterbalancing

Transfer effects (order effects)

Definition:
When taking part in earlier trials changes performance in the later trials

Types
Learning Fatigue Range or context effect

Problem:
Makes within-subjects designs difficult to interpret

Counterbalancing

Adjust condition order to unconfound transfer effects with condition effects

A,B,C A,C,B B,A,C B,C,A C,A,B C,B,A

Counter-balancing either withinor between- subjects

Between:
Joe: A,B Mary: B,A

Within:
Joe: ABBA Mary: ABBA

Things to worry about in counter-balancing

If within-subjects counter-balancing:
Linear transfer effects?
Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?
E.g., sometimes most learning happens in 1st trials

Always worry about asymmetrical transfer

Does A influence B more than B influences A?

Asymmetrical transfer
Quiet % trigrams remembered Noisy Time 1

Quiet

Noisy Time 2

Effect of noise depends on order People stick with the strategy they pick first
Or mix strategies

Partial counterbalancing: Latin Square

Every condition appears in every position equally:

Joe: A Mary: C John: B B A C C B A

Matching
Try to reduce between-group differences E.g., rank hearing as Good, Fair, Poor Unmatched, could get

Noisy: Poor1, Poor2, Fair1 Quiet: Good1, Good2, Fair2

Matched, get:
Noisy: Poor1, Fair2, Good1 Quiet: Poor2, Fair1, Good2

Matching
Match variable(s) and DVs should be strongly correlated Caveat: Match test should not affect DV

e.g., use existing match variable (SAT-M)

Note: Within-subjects designs match automatically

Number of levels

How many different groups or conditions that change just one independent variable
Two:
Experimental vs. control Massed vs. Distributed practice

More:
Drug vs. Placebo vs. No pill # of times an item is studied: 1,2,4,8, or 16 times

Inter- and extra-polating

60 40 20 0 0 1 2 3 # study repetitions 4

inside

outside

?
100 50 0 0 1 2 3 # study repetitions 4 60 40 20 0 0

1 2 3 # study repetitions

Floor & Ceiling Effects

Single Variable vs. Multiple Variables

Single Variable:
Only one independent variable Cannot look at interactions

Multiple Variables:
Two or more independent variables If use factorial design, can look at interactions Can require a lot of participants (between) or time (within)

Interactions
100 PrepLevel Manuscript Draft

% errors detected 0 Author Editor

Proofreader

Who finds more errors, author or editor? How to spot the interaction graphically?

Interactions
Two independent variables interact when the effect of one depends on the level of the other Independent vs. Control vs. Random

What if PrepLevel had been a control variable? What if PrepLevel had been a random variable? Make it an independent variable if there is reason to believe its effect might depend

Factorial design

Do all combinations of factors (cells)

E.g., Language learning

German Old Young French Old Young

Male

Female

Male

Female

A factor can be within or between

Converging Operations (converging series)

Using more than one method to test the same hypothesis

E.g., using experimental and observational methods E.g., using cross-sectional and longitudinal designs

Baseline procedure

Example 1: Clinical
No drug, drug, no drug, drug,...

Example

2: Education

Regular class, new format, regular class, new format,..