ANOVA & Comparisons

JACOB SEYBERT
10/01/09
 ANOVA Calculations

Hays and Brannick suggest slightly different

methods.
 Results in same conclusion.

From GLM 2 notes, see ANOVACalculations.xls:

 SStot   ( yij  y ) 2
G1 75 79 16
Control 77 79 4
M=79 79 79 0
SD=3.16 81 79 4
83 79 16
G2 80 79 1
M=84 82 79 9 Total
SD=3.16 84 79 25 Sum of
86 79 49 Squares
88 79 81
G3 70 79 81
M=74 72 79 49
SD=3.16 74 79 25
76 79 9
78 79 1
Sum 370
SSW   ( yij  y j ) 2 yij yj ( yij  y j ) 2
G1 75 79 16
Control 77 79 4
M=79 79 79 0
SD=3.16 81 79 4
83 79 16
G2 80 84 16
M=84 82 84 4 Within
SD=3.16 84 84 0 Sum of
86 84 4 Squares
88 84 16
G3 70 74 16
M=74 72 74 4
SD=3.16 74 74 0
76 74 4
78 74 16
Sum 120
SS B   n j ( y j  y ) 2 yj y ( y j  y)2
G1 79 79 0
Control 79 79 0
M=79 79 79 0
SD=3.16 79 79 0
79 79 0
G2 84 79 25
M=84 84 79 25
Between
SD=3.16 84 79 25
Sum of
84 79 25
Squares
84 79 25
G3 74 79 25
M=74 74 79 25
SD=3.16 74 79 25
74 79 25
74 79 25
Sum 250
ANOVA Source (Summary) Table

Source SS df MS F

Between 250 J-1= SS/df F=

Groups 3-1=2 250/2= MSB/MSW =
125 =MSB 125/10
=12.5
Within 120 N-J= 120/12 =
Groups 15-3=12 10 =
MSW

Total 370 N-1=

15-1=14 F( .05, 2,12)  3.89
 ANOVA

What you have done so far:

Proc GLM DATA = Data1;
CLASS Condition;
MODEL retention=condition;
Run;

This tells us if there are differences between the

groups.
What if we want to know where those differences
are?
 Comparing Means

2 options:
 Post-hoc tests
 Planned comparisons

Post-hoc tests evaluate differences between the

groups after a significant F value has been found.
Planned comparisons replace the main effect F value
and instead test specific hypotheses.
 Post-Hoc Tests

Many different methods of performing post-hoc

tests.
 Bonferroni
 Scheffe
 Tukey HSD
 REGWQ
 Post-Hoc Tests

Tukey's Studentized Range for honest significant

difference (HSD) Test: Controls the Type I family
wise error (FEW) rate, has power advantage—
when compare all possible pairs).

Bonferroni t Test: Controls the Type I family wise

error rate, but overcorrects for Type I error.

Scheffe's Test (modified F-test): Controls the Type

I family wise error rate, but over conservative.
 Post-Hoc in SAS

SAS template code:

PROC GLM;
CLASS categorical_variable;
MODEL dv=iv;
MEANS categorical_variable /TUKEY SCHEFFE
BON;

Can do all 3 at a time for comparison, or just do one.

 Example 1!

Import Data: ANOVAExample1.xls

Code:
 Example 1 Output:

Typical ANOVA output:

 Post-Hoc Test Output:

Tukey
 Post-Hoc Test Output:

Bonferroni
 Post-Hoc Test Output:

Scheffe
 Post-Hoc Differences

Power: Tukey>Bon>Scheffe
How about type I error then?
 Also Tukey>Bon>Scheffe
 Basically, Tukey is the most powerful but also has the
largest type I error.
 That is, Tukey is the most advantageous in terms of
power but the least advantageous in terms of avoiding
type I error.
Planned Comparisons
 Learning Suggestions

Brief overview here

Lecture provides details

Read Hays 423-467

Watch online example

 Planned Comparisons

Substitutes for an overall ANOVA test.

 Provides for the specific test of group comparisons:
 You decide these prior to the study

Is group 1 different than groups 2 & 3?

Is groups 1-3 different than groups 4-6?
 Planned Comparisons

Weights are applied to group means based on the

question being addressed:
A1 A2 A3 A4
Comparison 1 1/2 1/2 -1/2 -1/2
2 1 -1 0 0
3 0 0 1 -1

 Mean of the first two groups with the mean of the last two
groups.
 First two groups compared to each other.
 Last two groups compared to each other.
 Weights have to add to zero!
 Planned Comparisons

Comparisons must be independent linear

combinations with normal distributions and equal
variances.
 Then they are said to be Orthogonal
There are only J-1 orthogonal comparisons.
Can test for this by seeing if the products of the
weights assigned to each sum to zero.
 Pay attention in class for more detail!
 Planned Comparisons in SAS

PROC GLM;
CLASS categoricalvar;
MODEL DV = IV;
CONTRAST “Contrast Title1” categoricalvar 1 -3 1 1;
CONTRAST “Contrast Title2” categoricalvar 0 0 -1 1;
 Planned Comparison Example

Import Data: ANOVAExample1.xls

Code:
Planned Comparison Output
 Planned Comparison & Post Hoc

Can perform post-hoc tests with planned

comparisons.
 Problematic due to high Type I error.
 Not recommended.
PROC GLM data=d1;
Class rewgrp;
Model commit=rewgrp;
CONTRAST '2 vs 1&3' rewgrp .5 -1 .5;
Means rewgrp/Tukey Scheffe Bon Hovtest;
Run;
 Another Example!

Import Data: BrannickData.sas

IV “rewgrp”
 1 = low-reward condition
 2 = mixed-reward condition
 3 = high-reward condition
DV “commit” scale that measures employee
commitment (Range 0 to 36)
 Example3 Code:

Add in the following code:

PROC GLM data=d1;
Class rewgrp;
Model commit=rewgrp;
CONTRAST '2 vs 1&3' condition .5 -1 .5;
Means rewgrp/Tukey Scheffe Bon Hovtest;
Run;
What did you find?