Analysis of Variance

Analysis of Variance

• One-way ANOVA
• Two-way ANOVA
 Analysis of Variance

Analysis of Variance
One-way analysis of variance (One-Way ANOVA) is a method used
to compare 2 or more group means simultaneously.

Example:

The mean grades in Math 101 of CS students, BioChem

students, and APhy students are all equal

There are 2 variables here:

1. independent variable (qualitative): course
2. dependent variable (quantitative): Math 101 grade

The independent variable defines the different groups (populations)

according to the categories of that variable. If the variable has 3
or more categories, we use ANOVA to analyze the data.
 Analysis of Variance

Analysis of Variance: Hypotheses

Analysis of variance (ANOVA) is an extension of the t-test of two
independent samples.
ANOVA is for three or more independent samples.

Ho: μ1 = μ2 = … = μk. The means of the k populations are all

equal.

H1: μi ≠ μj for i ≠ j. At least one of the means is different.

At least two means are not equal.
The means are not all equal.

Terminology:
Factors are the independent variables (qualitative) in the study, e.g. Sex
Levels are the categories of the variable, e.g. Sex: Male/Female
Response is the dependent variable (quantitative)
 Analysis of Variance

Analysis of Variance: Rationale

Small differences between sample means are usually present.

The objective is to determine whether these differences are

significant. In other words, is the difference more that what might be
expected to occur by chance?

If the difference is more than what might be expected to occur by

chance, you have sufficient evidence to conclude that there are
differences between the population means.

- the differences are significant

 Analysis of Variance

Data Format

From each
population,
samples of
observations are
taken,
xij is the jth
observation from
the ith sample
(i=1,2,…,k)

Organize the data

as shown.
 Analysis of Variance

Analysis of Variance: Assumptions

1. Samples from each group are independent.

2. Dependent variable is normally distributed from each population.

Shapiro-Wilk’s test
Ho: Distribution is normal.
H1: Distribution is not normal.

3. Variances of the dependent variable are the same across populations

(homogeneity of variance ).

Barlett’s test
Ho: Population variances are all equal.
H1: Not all population variances are equal.
 Analysis of Variance

Analysis of Variance: Sum of Squares

ANOVA analyzes the variances of the data to determine whether there
is a difference between the group means.

The total variation, the overall variability in the response variable or

total sum of squares is partitioned into two parts:

Between Group Variation (model sum of squares) is the

variability explained by the
independent variable.

Within Group Variation is the variability not explained by the

independent variable. It is called the
error sum of squares.
 Analysis of Variance

One-Way ANOVA Table

Source df SS MS F
Between Group k-1 SSB MSB
MSB/MSW
Within Group n-k SSW MSW
Total n-1 SST

GT = grand total
 Analysis of Variance

One-Way ANOVA Table

Source df SS MS F
Between Group k-1 SSB MSB
MSB/MSW
Within Group n-k SSW MSW
Total n-1 SST

MSW = MSE
 Analysis of Variance

Decision and Conclusion

Reject Ho if Fcomputed is greater than
or equal the Ftabulated.

Ho is not rejected → There are no significant differences

among the sample means.
There is no sufficient evidence to
indicate that there are differences in the
population means.
.

Ho is rejected → At least one sample mean is

significantly different from the rest (at least
two means are significantly different)
.

There is sufficient evidence to indicate that at least two

population means are different.
 Analysis of Variance

One-Way ANOVA: Example 1

A researcher wants to test a new 100

0 mg 50 mg
anti-anxiety medication. They split mg
participants into three conditions
(0mg, 50mg, and 100mg), then ask 9 7 4
them to rate their anxiety level on a 8 6 3
scale of 1-10. Are there any 7 6 2
differences between the three
conditions? Use ɑ = 0.05. 8 5 3
8 8 4
9 7 3
8 6 2
Ho: μ0mg = μ50mg = μ100mg : The true mean anxiety level of the three
groups are all equal.
H1: Not all μ’s are equal: At least one mean anxiety level of the three
groups is different.
 Analysis of Variance

One-Way ANOVA: Example 1

Normality assumption:

Homogeneity assumption:
 Analysis of Variance

One-Way ANOVA: Example 1

Statistical
Decision:
Reject Ho
(p-value< 0.001).

Conclusion:
At least one of the
three conditions
differed
significantly on
anxiety level.
 Analysis of Variance

One-Way ANOVA: Example 2

Test scores in the entrance examination of
the incoming freshmen nursing students from
four different schools (A, B, C, and D) at
University X are shown below.

School Ho: μ1 = μ2 = μ3 = μ4
A B C D There are no differences in
the mean entrance exam
50
35 scores of the nursing
40 47
50 45 students across schools.
30 45
45 50
25 48
48 26 H1: Not all μ’s are equal
27 38
50 36
31 45 At least one the means is
24
39 different.
Are there differences in the mean scores?
 Analysis of Variance

One-Way ANOVA: Example 2

Normality assumption:

Homogeneity assumption:
 Analysis of Variance

One-Way ANOVA: Example 2

Statistical Decision:
Reject Ho. (p-value=0.0022).

Conclusion:
On the basis of these
data, we can conclude
that at least one school
differed significantly in
the mean entrance exam
score of nursing
students.
 Analysis of Variance

Types of ANOVA
One-Way ANOVA
• one factor (independent variable-qualitative) with at least two levels
• the observations across levels are independent (different subjects)

Example:
0 mg 50 mg 100 mg

A researcher wants to test a new 9 7 4

anti-anxiety medication. They split 8 6 3
participants into three conditions 7 6 2
(0mg, 50mg, and 100mg), then ask 8 5 3
them to rate their anxiety level on a 8 8 4
scale of 1-10. Are there any 9 7 3
differences between the three 8 6 2
conditions? Use ɑ = 0.05.
Response: Anxiety rating
Factor: Dosage
 Analysis of Variance

Types of ANOVA
Two-way ANOVA
• two factors (independent variable-qualitative)
• the observations across levels are independent (different subjects)

Example: Dosage
A study to determine the Center
effects of 3 doses of a new Placebo 25 mg 50 mg 75 mg
therapeutic agent on a 5 3 6 8 8 8 7 8
short-term memory 1 5 3 7 10 9 9 8 7
function was conducted at
two different centers. The
6 4 9 9 10 10 6 8
subjects were administered
2 5 8 8 6 8 10 9
a single oral dose of test
preparation and then
2 4 2 6 10 10 8 7 10
asked to recall items one 3 4 9 8 7 7 11 7
hour after exposure to a list
consisting of 12 items. Response: Anxiety rating
Factors: Dosage and Center
 Analysis of Variance

One-Way vs Two-Way ANOVA

One-Way ANOVA:
Ho: μ1= μ2= .... = μk Main effect
H1: Not all means are equal.

Two-way ANOVA:

Ho1: μA1= μA2= ... = μAk

H11: Not all means are equal.

Ho2: μB1= μB2= ... = μBl

H12: Not all means are equal.

Ho3: Interaction absent Interaction Effect

H13: Interaction present.
 Analysis of Variance

Main Effect in ANOVA

Ho: μA1= μA2= μA3

Example:
H1: Not all means are equal
Suppose we are
comparing the scores of
people who have
received medication
(100mg dosage group)
and people who have not
received medication (0mg
dosage group). The 0mg
condition has a mean of
60, while the 100mg has
a mean of 80.
 Analysis of Variance

Interaction Effect in ANOVA

Example:
Dosage: 0mg and 100mg)
Gender: men and women

In the 0mg dosage condition,

men have a mean of 60 while
women have a mean of 80.
.

In the 100mg dosage condition,

men have a mean of 80 while
women have a mean of 50.

Ho1: μA1= μA2= μA3 Ho2: μB1= μB2= μB3 H03: Interaction absent
H1: not all are equal H13: Interaction present
 Analysis of Variance

Assumptions
1. Normality. Each set of data has a normal distribution.

2. Homogeneity of variance. Different sets of data have equal

variances.

3. Sphericity. Differences in measurements between any two

variables are similar to differences between any other two.
 Analysis of Variance

Two-Way ANOVA
Source df SS MS F
Column Means
(A)
Error (AxS) dfA SSA MSA
dfE(AxS) SSE(AxS) MSE(AxS)
Row Means (B) dfB SSB MSB MSB/MSE
Error (BxS) dfE(BxS) SSE(BxS) MSE(BxS)
Interaction (AxB) dfAxB SS(AxB) MS(AxB)
dfE(AxBxS) SSE(AxBxS) MSE(AxBxS)
Error (AxBxS) dfE(s) SSE(S) MSE(S)
Error (S)
Total N-1 SST
dfA = a – 1 df(AxB) = (a – 1)(b – 1)
dfE(AxS) = (a – 1) (n – 1) dfE(AxBxS) = (a – 1)(b – 1)(n – 1)
dfB = b – 1 dfE(S) = (n – 1)
dfE(BxS) = (b – 1) (n – 1) dfT = N - 1
 Analysis of Variance

Two-Way ANOVA
 Analysis of Variance

Two-Way ANOVA
 Analysis of Variance

Two-Way ANOVA
Example: Researchers want to
compare anxiety levels of six
individuals at two marital status: Week
after they have divorced, and then Marital Subjec
Status t 1 2 3
again after they have gotten
married. Anxiety is measured three 1 5
3 7
times: week1, week2, and week3. 2
4
4
8
Anxiety is rated on a scale of 1-10, 3 3
Divorce 5 7
with 10 being “high anxiety” and 1 4 5
d 3 8
5 5
being “low anxiety”. Use ɑ = 0.05 6
4
5
7
level of significance to conduct an 3 7
analysis. 1
5 5 9
2
4 4 8
1st dependent factor: week 3
3 3 9
Married 4
5 5 8
2nd dependent factor: marital 5
5 5 7
6
status 4 4 9
 Analysis of Variance

Two-Way ANOVA

Marital Subjec Week

Status t Statistical Hypotheses:
1 2 3
1
3
5
7
1. Ho1: μdivorsed = μmarried
2
4
4
8 H11: μdivorsed ≠ μmarried
3 3
Divorce 5 7
4 5
d
5
3
5
8 2. Ho2: μweek1 = μweek2 = μweek3
6
4
5
7 H12: Not all μ’s are equal.
3 7

1 3. Ho3: An interaction is
5 5 9
2 absent.
4 4 8
3 H13: An interaction is
3 3 9
Married 4 present.
5 5 8
5
5 5 7
6
4 4 9
 Analysis of Variance

Two-Way ANOVA
 Analysis of Variance

Two-Way ANOVA

Statistical Decision:

Reject Ho1 at the 0.05 level of significance

(F=9.00>6.61, p=0.0301).

Reject Ho2 at the 0.05 level of significance

(F=92.19>4.10, p=0.0000).

Reject Ho3 at the 0.05 level of significance

(F=17.76>4.10, p=0.0005).
 Analysis of Variance

Two-Way ANOVA

Boxplot of anxiety level of the 3 pts in time

(after 1wk, after 2wks, after 3 wks)
 Analysis of Variance

Two-Way ANOVA

Boxplot of anxiety level (divorced vs remarried)

 Analysis of Variance

Two-Way ANOVA
Conclusion:
Anxiety levels
differed
significantly for
divorced and
remarried
individuals.
There was also
significant
difference between
the points in time
(after 1wk, 2 wks,
and 3wks).
An interaction effect
was also present.
 Sample R Implementation
One-way ANOVA
y <- c(2,3,1,2, 8,7,9,8, 11,12,13,12)
a <- gl(3,4) # or use “factor”
anova(lm(y ~ a))
 Sample R Implementation
Two-way ANOVA:
facmolb <- factor(ALLBm$mol.biol)
facB <-factor(ALLBm$BT)
anova(lm(ALLBm) ~ facB * facmolb))
 Sample R Implementation
Test for normality of residuals:
shapiro.test(residuals(lm(y ~ ALLB123$BT)))
#Ho: The population is normal.

Test for homogeneity of variance of residuals:

(Breusch-Pagan test)
library(lmtest)
bptest(lm(y ~ ALLB123$BT),studentize = FALSE)
#Ho: Variance is homogeneous.
 Analysis of Variance

References

Daniel, W. W. (1996). Biostatistics: A Foundation for Analysis in the

Health Sciences, 6th Edition. Wiley & Sons, Inc., USA.

Pagano, M. & Gauvreau, K. (2000). Principles of Biostatistics, 2nd Ed.

Duxbury Press.

Rosner, Bernard K. (2010). Fundamentals of Biostatistics , 7th Edition.

Brooks/Cole, Cengage Learning, Inc. USA.