Lecture 5

Tests of Differences
Between Means II
ANOVA
ANCOVA
BPK 304W
Fall 2022
 Announcements
• Lab 3 –Analysis for Writing Assignment #3 (due Wed,
Sept 28)
– Usual Canvas assignment quiz submission for results
– I will provide correct SPSS output after so that you can write
your results section of your project

• Assignment #1 (due Oct 14 )

– Will go over today
 Outline
• Review of t-tests
• ANOVA
• ANCOVA
• Assignment #1
Summary
 Learning Checkpoint
Independent or Paired t-test?
1. You want to compare systolic blood pressure while
sitting to systolic blood pressure 1-minute after
standing up in 75 stroke patients.
2. You want to determine if there is a difference in
standing long-jump between 10 year-old boys (n=43)
and girls (n=41).
3. You want to determine if head girth of physics majors
(n=50) is different than head girth of BPK majors
(n=50).
4. You want to test if foot length is the same as forearm
length among a sample of 76 football players.
 Learning Checkpoint
Independent or Paired t-test?
1. You want to compare systolic blood pressure while
sitting to systolic blood pressure 1-minute after
standing up in 75 stroke patients. Paired
2. You want to determine if there is a difference in
standing long-jump between 10 year-old boys (n=43)
and girls (n=41). Independent
3. You want to determine if head girth of physics majors
(n=50) is different than head girth of BPK majors
(n=50). Independent
4. You want to test if foot length is the same as forearm
length among a sample of 76 football players. Paired
 Tests of Differences Between
Means
Today’s theme: How do we determine if there is a
difference between two or more means?
1. t-test (last week)
– Compare 2 means from relatively small samples
– Independent t-tests and paired t-tests
2. ANOVA - Analysis of Variance (this week)
– Compare ≥ 2 means
– One-way, Two-way, Three-way
– Randomized groups, Repeated measures
3. ANCOVA - Analysis of Covariance (next
week)
– Compare ≥ 2 means and control for covariates (next week)
 Analysis of Variance -
ANOVA
When is ANOVA used?
– More sophisticated t-test
– Used to test for differences among multiple (>2) means
– Also used to test for differences between 2 means when
sample size is large (total n > 120)

Why is ANOVA used?

– More sophisticated t-test
– It’s useful because multiple t-tests result in increased chance of
type 1 error

How is ANOVA used?

– F (ratio) statistic is calculated and is evaluated in comparison to
the critical value of the F (ratio) statistic
Important Terminology for ANOVA
A FACTOR is an independent variable (e.g., experimental
condition) with a certain number of LEVELS.
– Sex is a factor that has 2 levels (male/female)
– Exercise grouped as aerobic/resistance/flexibility is a factor with 3
levels

• A between-subjects factor identifies different groups

– For example, AGE (young/old) is a between-subjects factor
with 2 levels.

• A within-subjects factor or repeated measures

factor identifies different conditions experienced by
one group
– For example, assume each participant completes 3 walking
conditions (level/uneven/incline). Then WALK CONDITION is
a within-subjects factor with 3 levels.
 One-way ANOVA
• One FACTOR (hence, one-way)
– HO: There is no difference in means
– HA: At least one mean is different

• Factors can have 2 or more LEVELS

- Sex: male, female (2 levels)
- Physical activity intensity: low, moderate, vigorous
(3 levels)
 Randomized Groups vs.
Repeated Measures ANOVA
• If we are dealing with only between-subjects factors,
we use Randomized Groups (RG) ANOVA.
– Similar to the independent t-test.

• If we are dealing with only within-subjects factors, we

use Repeated Measures (RM) ANOVA.
– Similar to the paired t-test.

• If we are dealing with both between-subjects factors

and within-subjects factors, we call it a Mixed
ANOVA.
F (ratio) Statistic - RG ANOVA
• Figure depicts scores from 4 groups.
• Red arrows illustrate variability within
each group.
• Black arrows illustrate variability between
groups (i.e., how much each group mean
varies from the overall grand mean)
• The F (ratio) statistic compares these two
sources of variability in the scores.
• The variability between the group means,
called Between Group Variability, is
compared with the variability among
individual scores within each of the
groups, called Within Group Variability.
• F (ratio) statistic increases as the
between group variability increases and
the within group variability decreases. Grand Mean
 Formula for Sources of
Variability

SS = sum of squares
-used for evaluating the variance of the data set from its mean
Individual participants are indexed by i (ranging from 1 to nj)
Groups of participants are indexed by j (ranging from 1 to p).
If there were 2 groups and each group had 10 participants, then i would range from
1 to 10, and j would range from 1 to 2.
 RG ANOVA Summary Table
SS df MS F

Between SS(Between) MS(Between)

SS(Between) k-1
Groups k-1 MS(Within)

Within SS(Within)
SS(Within) N-k
Groups N-k

SS(Within) +
Total N-1 .
SS(Between)

k=number of groups, N=total number of subjects, MS=mean square

F (ratio) statistic increases as between group variability increases and
within group variability decreases.
 Assumptions for RG ANOVA
• The populations from which the samples
were obtained are approximately normally
distributed.
• The samples are independent.
• The population value for the standard
deviation between individuals is the same in
each group.
– If standard deviations are unequal, transformation
of values may be needed before running a RG
ANOVA.
CFS Kids 17 – 19 years males)

p value

l One-way RG ANOVA
l Dependent variable: VO2max
l One factor with 3 levels: Age (17, 18, 19 years)
l No significant difference in mean VO2max between age groups in boys (p=0.102)
 CFS Kids 17 – 19 years females)

p value

l One-way RG ANOVA
l Dependent variable: VO2max
l One factor with 3 levels: Age (17, 18, 19 years)
l Significant difference in mean VO2max between age groups in girls (p=0.007). But where?
 Post Hoc Testing
• What? Post hoc simply means that the test is a
follow-up test done after the original ANOVA is found
to be significant.

• Why? If you are comparing 3+ means and find a

significant difference with ANOVA, post hoc tests
allow you to determine which means are different.

• How? You can do a series of comparisons, one for

each two-way comparison of interest.

• E.g. Bonferroni, Scheffe, or Tukey’s post hoc tests.

• The Bonferroni and Scheffe tests are conservative

and therefore are reasonable choices.
Scheffe’s Post Hoc Test
Males

• Males – no significant difference

from ANOVA, so you would not
run a post hoc test

Females

• Females – VO2max is
significantly lower for
19 year old females
than 17 year old
females (p=0.008)
 Learning Checkpoint
You are conducting an experiment to determine the
effects of education on income level. You recruit 3
groups of individuals based on highest level of education
attained: high school graduation; bachelor’s degree;
graduate degree. You ascertain their average annual
income over the past 3 years.
1. What is the dependent variable?
2. What is the independent variable/factor?
3. How many levels does the factor have, and what are
they?
4. What statistical test would you use to compare mean
levels of the dependent variable?
 Learning Checkpoint
You are conducting an experiment to determine the effects of
education on income level. You recruit 3 groups of individuals
based on highest level of education: high school graduation,
bachelor’s degree; graduate degree. You ascertain their
average annual income over the past 3 years.
1. What is the dependent variable? Income
2. What is the independent variable/factor? Education
3. How many levels does the factor have, and what are
they?
3 levels (high school grad., bachelor’s degree, grad. degree)
4. What statistical test would you use to compare mean
levels of the dependent variable? One-way RG ANOVA. If
p<0.05, post-hoc.
 RG ANOVA - Multiple Factors
• Test of differences between means with two or
more grouping factors, such that each factor is
‘adjusted’ for the effect of the other.

• Used to evaluate significance of individual

factors (main effects) and interactions between
the factors.
 RG ANOVA - Multiple Factors
• For 2-way ANOVA, there will be:
– Two factors (main effects) considered simultaneously: A & B
– One 2-way interaction (AxB)
Example: 2-Way RG ANOVA
Dependent variable: VO2max
2 Grouping Factors: AGE (17, 18, 19) and Runner type (Long distance runner, Sprinters)

Long D Sprinter

Long D
Sprinter

Long D
Sprinter

Long D
Sprinter

Long D
Sprinter How would you describe the data
shown in this graph?
 Example: 2-Way RG ANOVA cont.
Dependent variable: VO2max
2 Grouping Factors: AGE (17, 18, 19) and Event (Long distance runner, Sprinters)

Runner type

Runner type

RESULTS
• Main effect: Significant difference in VO2max (p=0.018) by AGE. Do post hoc.
• Main effect: Significant difference in VO2max (p<0.001) by Runner types. No post
hoc.
• No significant AGE * Runner types interaction (p=0.386) on VO2max. How would you
interpret this in words?
Example: 2-Way RG ANOVA cont.
Dependent variable:
Dependent variable: VO
VO22max
max
2 Grouping Factors: AGE Factors:
2 Grouping (17, 18, 19)
AGE and Event
(17, (Long
18, 19) anddistance
SEX (1, runner,
2) Sprinters)

Bonferroni post-hoc on AGE

 Example: 2-Way RG ANOVA cont.
Dependent variable:
Dependent variable: VO
VO22max
max
2 Grouping Factors: AGE Factors:
2 Grouping (17, 18, 19)
AGE and Event
(17, (Long
18, 19) anddistance
SEX (1, runner,
2) Sprinters)

How would you interpret the results in words?

• We observed a significant effect of age on VO2max (F(2,897)=4.034, p=0.018). Post-hoc
testing revealed that, independent of runner type, VO2max was higher in 17 year olds
(mean=4.57, SD=0.93 l/min) than in 18 year olds (mean=4.40, SD=0.89 l/min) (p=0.002)
or 19 year olds (mean=4.25, SD= 0.91 l/min) (p<0.001), and it was also higher in 18 year
olds than in 19 year olds (p=0.013).
• We also observed a significant effect of runner type on VO2max
(F(1,897)=1074.742,p<0.001). Specifically, independent of age, VO2max was higher in
long distance runners (mean=5.07, SD= 0.77 l/min) than sprinters (mean=3.71, SD=0.37
l/min) (p<0.001).
• There was no interaction between age and runner type on VO2max (F(2,897)=0.952,
p=0.386), indicating that the age-related decline in VO2max observed between the ages
of 17 and 19 years was similar for long distance runners and sprinters.
 Grad Student Joe
• Joe wants to determine the effect of flooring on the hand forces required
to push a sit-to-stand lift among long-term care workers.
• He has a validated method to measure push forces.
• He decides to test the effects of concrete (1), carpet (2), and anti-fatigue
(3) flooring.
• He knows of 2 approaches to design his study:
(A) (B)

1 2 3 1 2 3
Flooring Flooring
28
 Grad Student Joe
• What are the advantages & disadvantages of each approach?
• Which approach should Joe pursue, A or B?
• Breakout rooms for 5 mins. Identify someone to report back from your
group.
Repeated Measures Randomized Groups
(RM) Design (RG) Design
(A) (B)

1 2 3 1 2 3
Flooring Flooring
29
 Repeated Measures (RM)
ANOVA
• In a repeated measures design, the same dependent
variable is measured several times under different
experimental conditions for each participant.
• Pre- and post-test scores are the simplest RM design
example
– use paired t-test if there are just 2 means & sample size is
small.
– with 3 or more means or a large sample size, you use RM
ANOVA.
• RM designs have advantages over RG designs:
– eliminate variability due to differences between subjects, which
makes it easier to identify the effect of the independent
variable.
30
– often require fewer participants and less time
 1-Way RM ANOVA
Dependent variable: Heart rate (bpm)
1 Within-Subjects Factors: Position (supine, sitting, standing)

Heart rate was measured in 13 participants under 3

different positions: supine, sitting, and standing.

31
1-Way RM ANOVA cont.
Dependent variable: Heart rate (bpm)
1 Within-Subjects Factors: Position (supine, sitting, standing)

32
 1-Way RM ANOVA cont.
Dependent variable: Heart rate (bpm)
1 Within-Subjects Factors: Position (supine, sitting, standing)

• There was a significant effect of position on heart rate

(F(2,24)=29.32, p<0.001).
• Which positions caused a change in heart rate? Do post hoc test
because position has > 2 levels.
 1-Way RM ANOVA cont.
Dependent variable: Heart rate (bpm)
1 Within-Subjects Factors: Position (supine, sitting, standing)

• Post-hoc testing revealed that heart rate increased between supine (60.9
bpm) and sitting (71.8 bpm) positions (p=0.001) and between supine and
standing (76.5 bpm) positions (p<0.001), but not between sitting and
standing positions (p=0.069).
 1-Way RM ANOVA cont.
Dependent variable: Heart rate (bpm)
1 Within-Subjects Factors: Position (supine, sitting, standing)

35
* Significantly different from supine, p=0.001 for sitting, p<0.001 for standing
 2-Way RM ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
2 Within-Subjects Factors: Position (supine, presyncope), Time (1, 2)

Systolic arterial pressure was measured in 15 participants under 2 positions

(supine and presyncope) at 2 times.

36
2-Way RM ANOVA cont.
Dependent variable: Systolic arterial pressure (mm Hg)
2 Within-Subjects Factors: Position (supine, presyncope), Time (1, 2)

37
 2-Way RM ANOVA cont.
Dependent variable: Systolic arterial pressure (mm Hg)
2 Within-Subjects Factors: Position (supine, presyncope), Time (1, 2)

• We observed a significant effect of position on systolic arterial pressure (F(1,14)=195.75, p<0.001).

• Averaged over time, systolic arterial pressure declined between supine (115.7 mmHg) and presyncope
(69.5 mmHg) positions.
• Post hoc test is not necessary because there are only 2 means.
38
 2-Way RM ANOVA cont.
Dependent variable: Systolic arterial pressure (mm Hg)
2 Within-Subjects Factors: Position (supine, presyncope), Time (1, 2)

Averaged across positions, systolic arterial pressure did not change between the first (93.5 mmHg)
and second (91.6 mmHg) measurement (F(1,14)=0.33, p=0.577). In addition, there was 39 no significant
interaction between position and time (F(1,14)=1.57, p=0.231).
 2-Way Mixed ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
1 Between-Subject Factor: Health Status (Healthy, Frail)
1 Within-Subjects Factor: Time (1, 2).

Systolic arterial pressure was measured twice in 15 healthy and 15 frail subjects.

40
2-Way Mixed ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
1 Between-Subject Factor: Health Status (Healthy, Frail)
1 Within-Subjects Factor: Time (1, 2).

41
2-Way Mixed ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
1 Between-Subject Factor: Health Status (Healthy, Frail)
1 Within-Subjects Factor: Time (1, 2).

42
2-Way Mixed ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
1 Between-Subject Factor: Health Status (Healthy, Frail)
1 Within-Subjects Factor: Time (1, 2).

43
 2-Way Mixed ANOVA
Dependent variable: Systolic arterial pressure (mm Hg)
1 Between-Subject Factor: Health Status (Healthy, Frail)
1 Within-Subjects Factor: Time (1, 2).

• We observed a significant health status x time

interaction on systolic arterial pressure
(F(1,28)=87.47, p<0.001). [Consult graph to interpret
nature of interaction].
– While arterial pressure increased between the first and
second measurements in healthy adults, the effect was
opposite in frail adults who experienced a decline in arterial
pressure between measurements.
– Note, we do not interpret the main effects of health status or
time because the interaction was significant. In these
situations, the main effects are not meaningful (accurate).
44
 3-Way ANOVA
• Three factors considered simultaneously, e.g., 2 Group
(glaucoma, control) x 3 Lighting (normal, dim, sudden
change) x 2 Motion (stationary, in motion).
• Three main effects: Group, Lighting, Motion
• Three 2-way interactions: Group x Lighting, Group x
Motion, Lighting x Motion
• One 3-way interaction: Group x Lighting x Motion
• Interpretation of results from a 3-Way ANOVA is fairly
complex and beyond the scope of BPK 304W.

45
 Acknowledgments
• Data for some examples were provided by Dr.
Victoria Claydon.

46
 Analysis of Covariance
(ANCOVA)
Confounder
(W)

Independent Dependent
Variable (X) Variable (Y)

When you compare groups of individuals, there may be

important differences between the groups that influence
the dependent variable.

Fortunately, when you test for a difference between

means, you can take into account (or “adjust”) for a
relationship between the dependent variable and another
variable (covariate/confounder) with ANCOVA.
 ANCOVA
Example: If you want to test whether grip strength is
greater in weightlifters than runners, you might want to
consider that weightlifters tend to be have bigger
forearm muscles than sprinters. If you account for
differences in muscle size, is grip strength still different
between the different type of athletes? Let’s see.

Muscle size

Athlete type Grip strength

 Correlations between muscle size and
maximum grip strength for weightlifters vs
runners
Weightlifters
Runners

Muscle Size
 Independent t tests show that weightlifters have
greater skinfold-adjusted forearm girth (i.e., muscle
size) and grip strength than runners.

Athlete type W Lift = 1, Runner = 2

W Lift
Runner

W Lift
Runner

Are weightlifter grip strength greater just because they have bigger muscles? Or, are there
biological differences in weightlifters and runners muscle, rendering weightlifter muscle stronger
 ANCOVA
Dependent variable: Maximum Grip Strength
Fixed Factor: Athlete type
Covariate: Skinfold-Adjusted Forearm Girth (SAFAGRightcm)

Athlete type

• Skinfold-adjusted forearm girth (representing muscle

size) is a significant covariate.
• There is no significant difference between athlete type
in grip strength after ‘adjusting’ for muscle size.
 Review: What is a 2-way ANOVA
• A 2-way ANOVA is used when you are testing the effects of
2 independent variables (factors) on a dependent variable.
• For example, what are the effects of age and caffeine intake
on resting heart rate?
• With a 2-way ANOVA you can determine the main effects of
each independent variable
– How does age affect resting heart rate, independent of caffeine
intake?
– How does caffeine intake affect resting heart rate, independent of
age?

• You can also determine if there is an interaction between

the two independent variables on the dependent variable.
– Does resting heart rate change with caffeine intake the same way in
older and younger adults?
 Review: What is a Mixed
ANOVA?
• A mixed ANOVA is a specific type of 2-way
ANOVA where one factor is a between-
groups factor and one factor is a within-
groups (repeated measures) factor.
• For example, you tested gait speed over 3
different distances (short, medium, long) in
young and elderly adults
• You have:
– one between-groups factor = age
– one within-subjects factor = walk distance
 Review: What is a “Post Hoc”
Test?
• If you find a significant main effect from an ANOVA for
a factor that has 3 or more levels, you do not know
initially which means are different from one another.
• You can use a post hoc test to determine which
means are different from one another.
• Post hoc tests are so named because they are
completed after one observes a significant main
effect or interaction.
 Review: What is a Statistical
Interaction?
When you have 2 or more factors in a statistical model
(e.g., a 2-way ANOVA), a statistical interaction occurs
when that the effect of one factor depends on the value
of another factor.
With Spinal Cord Injury Without Spinal Cord Injury

Blood
Pressure

Pre Post Pre Post Pre Post

Hot Tub Hot Tub Hot Tub Hot Tub Hot Tub Hot Tub
NO interaction YES interaction YES interaction
 Review: Between-Group vs. Within-Group
Variability
Step Reaction Time Data (s)

Between-group variability (e.g., 0.06 s Here, between-group variability is the

between wrist and hip fracture patients) is
quite a bit larger than within-subject
variability (SD=0.01 s for all groups).
One-way ANOVA (F(2,27)=285.1.
same as in the example on the left, but
there is more within-group variability (SDs
of 0.03 & 0.04 s). Thus, there is more
overlap in reaction time across the groups.
One-way ANOVA (F(2,27)=13.1.
 Review: What Test Should You Use?
How many means are you comparing?

2 More than 2

Total sample Total sample 1 factor 2 factors 3 factors

size ≤ 120 size > 120 (A) (A,B) (A,B,C)

One-way Two-way Three-way

Means from 2 Means from 1 Means from 2 Means from 1 ANOVA ANOVA ANOVA
independent group independent group
groups (i.e., paired) groups (i.e., paired)
Main effects:
Main effects:
A,B, and C
Main effect: A, B
Interactions:
Independent One-way RG One-way RM A Interactions:
AxB, AxC,
Paired t-test AxB
t-test ANOVA ANOVA BxC, AxBxC

If only between-groups factors: RG ANOVA

If only within-groups factors: RM ANOVA
If between-groups & within-groups factors: Mixed

Abbreviations: RG, randomized groups; RM, repeated measures

 3-Way ANOVA
• Three factors considered simultaneously:
– Group (glaucoma, control)
– Lighting (normal, dim, sudden change) How many
– Motion (stationary, in motion) means?
• Three main effects: Group, Lighting, Motion
• Three 2-way interactions: Group x Lighting, Group x
Motion, Lighting x Motion
• One 3-way interaction: Group x Lighting x Motion
• Interpretation of results from a 3-Way ANOVA is fairly
complex and beyond the scope of BPK 304W.
 Learning Checkpoint ✅
You wish to test push and pull forces required to
move a laundry cart using 4 different types of
castors in a group of 35 hospital employees.
What are the DVs? Factors? # of levels of the
factors? What type of analysis would you use?
 ✅
Learning Checkpoint
You wish to test push and pull forces required to
move a laundry cart using 4 different types of
castors in a group of 35 hospital employees. What
are the DVs? Factors? # of levels of the factors?
What type of analysis would you use?
DVs: push forces, pull forces
Factor: Castor (4 levels)
Analysis: You would use 1-way repeated
measures ANOVA for each DV. If there was
a significant main effect for Castor, you
would do a post-hoc test to determine where
the differences were.
 Learning Checkpoint✅
You wish to test push forces required to move a
laundry cart using 4 different types of castors in
35 hospital employees and 35 hotel employees.
What are the DVs? Factors? # of levels of the
factors? What type of analysis would you use?
 Learning Checkpoint ✅
You wish to test push forces required to move a
laundry cart using 4 different types of castors in 35
hospital employees and 35 hotel employees. What
are the DVs? Factors? # of levels of the factors?
What type of analysis would you use?
DVs: push forces
Factors: Castor (4 levels), Job Type (2 levels)
Analysis: You would use a 2-way mixed
ANOVA (Castor is a repeated measures
factor; Job Type is a between-groups factor).
Look for a significant Castor x Job Type
interaction.