Two-way MANOVA

• We now consider designs with two factors. Factor 1 has g

levels and factor 2 has b levels.
• If Xikr is the p × 1 vector of measurements on the rth unit
in the ith level of factor 1 and the kth level of factor 2:
Xikr = µ + αi + τ k + γ ik + eikr ,
with i = 1, ..., g, k = 1, ..., b and r = 1, ..., n and all are p × 1
vectors.
• Here γik is a vector of interaction effects.
• To simplify notation, we assume that there are n units in
each of the gb combinations of factor levels, but everything
we will say holds as well for the case where there are ngb units
in each factor combination.

420
 Two-way MANOVA

• The vector of measurements taken on the r-th unit in the

treatment group distinguished by the i-th level of factor 1
and the k-th level of factor 2 can be expressed as
           
Xikr1 µ1 αi1 τk1 γik1 ikr1
 Xikr2   µ2   αi2   τk2   γik2   ikr2 
= + + + +
           

 ...   ...   ...   ...   ...   ... 

Xikrp µp αip τkp γikp ikrp
where
ikr ∼ N IDp(0, Σ)

421
 Two-way MANOVA

• We will impose the SAS restrictions:

αg` = 0 for ` = 1, 2, ..., p
τb` = 0 for ` = 1, 2, ..., p
γgk` = 0 for k = 1, 2, ...b and ` = 1, 2, ..., p
γib` = 0 for i = 1, 2, ...g and ` = 1, 2, ..., p

• By making the responses from different units the rows of a

response matrix, the model for the entire set of data can be
written in matrix notation as follows

422
 Two-way MANOVA
 
X1111 X1112 ... X111p
 X1121 X1122 ... X112p 
=
 

 ... ... ... ... 
Xgbn1 Xgbn2 ... Xgbnp
 
  µl µ2 ... µp
1 1 0 ... 0  α11 α12 ... α1p 
1 1 0 ... 0
   
   ... ... ... ... 

 ... ... ... ... ...






   αg−1,1 αg−1,2 ... αg−1,p 
 1 1 0 ... 0
  
   τ11 τ12 ... τ1p 
1 0 1 ... 0 +
   
   ... ... ... ...

 ... ... ... ... 





   τb−1,1 τb−1,2 ... τb−2,p 
 1 0 1 ... 0   

 ... ... ... ...



 γ111 γ112 ... γ11p 

   ... ... ... ... 
1 0 0 ... 0
 
γg−1,b−1,1 γg−1,b−1,2 ... γg−1,b−2,p

423
 Two-way MANOVA

• If the interaction is non-zero, then factor effects are not ad-

ditive and the effect of one factor may depend on the level
of the other factor.

• See next figure: on top panel the effect of factor 2 depends

on level of factor 1. When factor 1 is at level 2, there rela-
tively little effect of factor 2 on the outcome variable.

• The bottom panel shows parallel profiles, an example with

no interaction

424
 Sum of squares and cross-product matrices

• As in the one-factor model, we can decompose the overall

variability into different sources. Note that

xikr − x̄ = (x̄i. − x̄) + (x̄.k − x̄) + (x̄ik − x̄i. − x̄.k + x̄) + (xikr − x̄ik ),
where x̄i. is the p × 1 mean vector of observations at ith level
of factor 1, x̄.k is the p × 1 mean vector of observations at
kth level of factor 2 and x̄ik is the p × 1 mean vector of ob-
servations at the ith level of factor 1 and the kth level of
factor 2.

• Multiplying both sides in expression above by the correspond-

ing transposed vectors and summing over n, k, i we get the
usual decomposition.

425
 Two-way MANOVA
Source SS and CP matrices Degress of freedom

Factor 1 SSPf ac1 g−1

Factor 2 SSPf ac2 b−1
Interaction SSPint (g − 1)(b − 1)
Residual SSE gb(n − 1)

Corr. total SSPctot gbn − 1

See next page for expressions for the various SS and CP matrices.

426
 Two-way MANOVA

bn(x̄i. − x̄)(x̄i. − x̄)0

X
SSPf ac1 =
i
gn(x̄.k − x̄)(x̄.k − x̄)0
X
SSPf ac2 =
k
n(x̄`k − x̄i. − x̄.k + x̄)(x̄ik − x̄i. − x̄.k + x̄)0
X X
SSPint =
i k
(xikr − x̄ik )(xikr − x̄`k )0
XX X
SSE =
i k r
(xikr − x̄)(xikr − x̄)0.
XX X
SPctot =
i k r

All matrices are p × p dimensional.

427
 Testing hypothesis in the two-way model
• We first test for the presence of absence of interaction
effects:
H0 : γ11 = γ12 = · · · = γgb = 0,
versus H1 : at least one γik 6= 0.

• The Wilk’s Λ statistic below has an asymptotic χ2 distribu-

tion. For
|SSE|
Λ= ,
|SSPint + SSE|
we reject H0 at level α if
" #
p + 1 − (g − 1)(b − 1)
− bg(n − 1) − ln Λ ≥ χ2
p(g−1)(b−1) (α).
2

428
 Testing hypothesis in the two-way model

• A more accuate p-value is obtained from Rao’s F-approximation

• If we reject the null hypothesis of no interaction effects,

then interpreting the meaning of factors 1 and 2 effects
gets complicated.

• One recommended approach in this case is to focus on the

p variables individually (perhaps by fitting the p univariate
ANOVA models) to see whether interactions are present on
all outcome variables or only on some of them.

429
 Testing hypothesis in the two-way model

• Interpretation of results can be aided by constructing p profile

plots (one for each of the p response variables) with the
sample mean at each combination of factor levels substituted
for the corresponding population mean.

• If we fail to reject the null hypothesis of no interaction effects,

then we proceed with hypothesis tests for additive effects
of factors 1 and 2 using the appropriate multivariate test
statistics.

430
 Testing hypothesis in the two-way model
(cont’d)

• The null hypothesis of no additive effect of factor 1 is

H0 : α1 = α2 = · · · = αg = 0,
versus H1 : at least one αil 6= 0. The Wilk’s Λ∗ statistic is
|SSE|
Λ∗ = ,
|SSPf ac1 + SSE|
and the null is rejected at level α if
" #
p + 1 − (g − 1)
− bg(n − 1) − ln Λ∗ ≥ χ2
p(g−1) (α).
2

431
 Testing hypothesis in the two-way model
(cont’d)
• Similarly, the null hypothesis of no additive effect of factor 2
is
H0 : τ1 = τ2 = · · · = τb = 0,
versus H1 : at least one τk 6= 0. The Wilk’s Λ∗ statistic is
|SSE|
Λ∗ = ,
|SSPf ac2 + SSE|
and the null is rejected at level α if
" #
p + 1 − (b − 1)
− bg(n − 1) − ln Λ∗ ≥ χ2
p(b−1) (α).
2

432
 Simultaneous confidence intervals

• As in the one-factor case, we may wish to explore differences

across factor levels for each of the p variables.

• For example, simultaneous 100(1 − α)% Bonferroni confi-

dence intervals for differences between the g(g − 1)/2 pairs
of levels of factor 1 (α`j − αmj ) for all j = 1, 2, ..., p response
variables are constructed as
s
α Ejj 2
(x̄`.j − x̄m.j ) ± tν ( ) ,
pg(g − 1) ν bn
where ν = gb(n−1), Ejj is the (j, j)th diagonal element of the
error SS and CP matrix, and (x̄`.j − x̄m.j ) is the jth element
of the p × 1 vector of sample mean differences (x̄`. − x̄m.).

433
 Simultaneous confidence intervals (cont’d)

• Similarly, simultaneous 100(1 − α)% Bonferroni confidence

intervals for differences between the b(b − 1)/2 pairs of levels
of factor 1 (τkj − τqj ) for all j = 1, 2, ..., p response variables
are constructed as
s
α Ejj 2
(x̄.kj − x̄.qj ) ± tν ( ) ,
pb(b − 1) ν gn
where ν = gb(n−1), Ejj is the (j, j)th diagonal element of the
error SS and CP matrix, and (x̄.kj − x̄.qj ) is the jth element
of the p × 1 vector of sample mean differences (x̄.k − x̄.q ).

434
 Simultaneous confidence intervals (cont’d)

• We need to consider combinations of factor levels if the in-

teraction effects are not negligible.
• For example, simultaneous 100(1 − α)% Bonferroni confi-
dence intervals for differences between the g(g − 1)/2 pairs
of levels of factor 1 ((τ`j + γ`kj ) − (τmj − γmkj )) at each of the
k = 1, 2, ..., b levels of factor 2 and all j = 1, 2, ..., p response
variables are constructed as
s
α Ejj 2
(x̄`kj − x̄mkj ) ± tν ( ) ,
pbg(g − 1) ν n
where ν = gb(n − 1), Ejj is the (j, j)th diagonal element of
error SS and CP matrix, and (x̄`kj − x̄mkj ) is the jth element
of the p × 1 vector of sample mean differences (x̄`k − x̄mk ).

435
 Some Comments

• Note: if n = 1, that is we do not have any replications within

factor level combinations, we will not be able to estimate an
error SS an CP matrix. In that case, we can only make for-
mal inferences about the additive effects model for factors 1
and 2.
• Note 2: The extension to designs with more than two factors
is straight forward.
• It is possible to fit interactions of third, fourth and higher
orders when three, four or more factors are included in the
experiment.
• Interpretations are complicated if higher order interactions
are not negligible.

436
 Example: Peanuts

• See Problem 6.31 in text book.

• Plant scientists conducted an experiment to examine three

traits of peanuts. The two factors in the experiment were
variety (three levels) and location (two levels), so there are
g × b = 3 × 2 = 6 factor level combinations.

• Two replications n = 2 were included for each of the 6 com-

binations of factor levels.

437
 Example: Peanuts

• Scientists measured three variables on each plot:

– X1 = yield (plot weight)

– X2 = weight in grams of sound mature kernels - 250

grams

– X3 = seed size measured as weight in grams of 100 seeds

• Fit a two-way model with an interaction using SAS.

438
 Example: Peanuts(cont’d)
• It is useful to first obtain the profile plots for the three
variables to see whether there may be interaction effects of
location and variety.

• We plot the means of each variable in each variety by

locations.

• See figures that follow. There are clear interactions between

location and variety, and the interactions appear to be
significant for all three variables.

439
 Example: Peanuts(SAS Code)

/* This program performs a two-way MANOVA

on the peanut data posted as peanuts.dat
The code is posted as peanuts.sas */

options linesize=64 nocenter nonumber ;

data set1;
infile "c:\stat501\data\peanuts.dat";
INPUT location variety x1 x2 x3;
/* LABEL group = student group
x1 = yield
x2 = SdMatKer
x3 = SeedSize; */
run;
440
 Example: Peanuts(SAS Code)

PROC PRINT data=set1; run;

PROC GLM DATA=set1;

CLASS location variety;
MODEL x1-x3 = location variety location*variety / P SOLUTION;
MANOVA H=location*variety /PRINTH PRINTE;
MANOVA H=variety / printH printE;
MANOVA H=location / printH printE;
Repeated traits 3 profile / printm;
run;

441
 Example: Peanuts(SAS Output)

Obs location variety x1 x2 x3

1 1 5 195.3 153.1 51.4

2 1 5 194.3 167.7 53.7
3 2 5 189.7 139.5 55.5
4 2 5 180.4 121.1 44.4
5 1 6 203.0 156.8 49.8
6 1 6 195.9 166.0 45.8
7 2 6 202.7 166.1 60.4
8 2 6 197.6 161.8 54.1
9 1 8 193.5 164.5 57.8
10 1 8 187.0 165.1 58.6
11 2 8 201.5 166.8 65.0
12 2 8 200.0 173.8 67.2
442
 Example: Peanuts(SAS Output)

Dependent Variable: x1
Sum of
Source DF Squares Mean Square
Model 5 401.9175000 80.3835000
Error 6 104.2050000 17.3675000
Corrected Total 11 506.1225000

Source DF Type I SS Mean Square F Value Pr > F

location 1 0.7008333 0.7008333 0.04 0.8474
variety 2 196.1150000 98.0575000 5.65 0.0418
location*variety 2 205.1016667 102.5508333 5.90 0.0382

443
 Example: Peanuts(SAS Code)

Dependent Variable: x2
Sum of
Source DF Squares Mean Square
Model 5 2031.777500 406.355500
Error 6 352.105000 58.684167
Corrected Total 11 2383.882500

Source DF Type I SS Mean Square F Value Pr > F

location 1 162.067500 162.067500 2.76 0.1476
variety 2 1089.015000 544.507500 9.28 0.0146
location*variety 2 780.695000 390.347500 6.65 0.0300

444
 Example: Peanuts(SAS Code)

Dependent Variable: x3
Sum of
Source DF Squares Mean Square
Model 5 442.5741667 88.5148333
Error 6 94.8350000 15.8058333
Corrected Total 11 537.4091667

Source DF Type I SS Mean Square F Value Pr > F

location 1 72.5208333 72.5208333 4.59 0.0759
variety 2 284.1016667 142.0508333 8.99 0.0157
location*variety 2 85.9516667 42.9758333 2.72 0.1443

445
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and F Approximations for the

Hypothesis of No Overall location*variety Effect
H = Type III SSCP Matrix for location*variety
E = Error SSCP Matrix
S=2 M=0 N=1

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.07430 3.56 6 8 0.0508
Pillai’s Trace 1.29086 3.03 6 10 0.0587
Hotelling-Lawley Trace 7.54429 5.03 6 4 0.0699
Roy’s Greatest Root 6.82409 11.37 3 5 0.0113

NOTE: F Statistic for Roy’s Greatest Root is an upper bound.

NOTE: F Statistic for Wilks’ Lambda is exact.
446
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and F Approximations for

the Hypothesis of No Overall variety Effect
H = Type III SSCP Matrix for variety
E = Error SSCP Matrix
S=2 M=0 N=1

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.01244 10.62 6 8 0.0019
Pillai’s Trace 1.70911 9.79 6 10 0.0011
Hotelling-Lawley Trace 21.37568 14.25 6 4 0.0113
Roy’s Greatest Root 18.18761 30.31 3 5 0.0012

NOTE: F Statistic for Roy’s Greatest Root is an upper bound.

NOTE: F Statistic for Wilks’ Lambda is exact.
447
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and Exact F Statistics for

the Hypothesis of No Overall location Effect
H = Type III SSCP Matrix for location
E = Error SSCP Matrix
S=1 M=0.5 N=1

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.10651620 11.18 3 4 0.0205
Pillai’s Trace 0.89348380 11.18 3 4 0.0205
Hotelling-Lawley Trace 8.38824348 11.18 3 4 0.0205
Roy’s Greatest Root 8.38824348 11.18 3 4 0.0205

448
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and Exact F Statistics

for the Hypothesis of no traits Effect
H = Type III SSCP Matrix for traits
E = Error SSCP Matrix
S=1 M=0 N=1.5

Statistic Value F Value Num DF Den DF Pr>F

Wilks’ Lambda 0.000089 28198.5 2 5 <.0001
Pillai’s Trace 0.999911 28198.5 2 5 <.0001
Hotelling-Lawley Trace 11279.385332 28198.5 2 5 <.0001
Roy’s Greatest Root 11279.385332 28198.5 2 5 <.0001

449
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and Exact F Statistics for

the Hypothesis of no traits*location Effect
H = Type III SSCP Matrix for traits*location
E = Error SSCP Matrix
S=1 M=0 N=1.5

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.11262 19.70 2 5 0.0043
Pillai’s Trace 0.88738 19.70 2 5 0.0043
Hotelling-Lawley Trace 7.87949 19.70 2 5 0.0043
Roy’s Greatest Root 7.87949 19.70 2 5 0.0043

450
 Example: Peanuts(SAS Code)

MANOVA Test Criteria and F Approximations for

the Hypothesis of no traits*variety Effect
H = Type III SSCP Matrix for traits*variety
E = Error SSCP Matrix
S=2 M=-0.5 N=1.5

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.02064 14.90 4 10 0.0003
Pillai’s Trace 1.55258 10.41 4 12 0.0007
Hotelling-Lawley Trace 19.67760 24.15 4 5.1429 0.0016
Roy’s Greatest Root 18.14717 54.44 2 6 0.0001

NOTE: F Statistic for Roy’s Greatest Root is an upper bound.

NOTE: F Statistic for Wilks’ Lambda is exact.
451
 Example: Peanuts(SAS Code)

Repeated Measures Analysis of Variance

MANOVA Test Criteria and F Approximations for the

Hypothesis of no traits*location*variety Effect
H = Type III SSCP Matrix for traits*location*variety
E = Error SSCP Matrix
S=2 M=-0.5 N=1.5

Statistic Value F Value Num DF Den DF Pr > F

Wilks’ Lambda 0.09547 5.59 4 10 0.0125
Pillai’s Trace 1.19307 4.44 4 12 0.0198
Hotelling-Lawley Trace 6.45248 7.92 4 5.1429 0.0204
Roy’s Greatest Root 5.94400 17.83 2 6 0.0030

NOTE: F Statistic for Roy’s Greatest Root is an upper bound.

NOTE: F Statistic for Wilks’ Lambda is exact.
452
 Example: Peanuts(SAS Code)

Repeated Measures Analysis of Variance

Tests of Hypotheses for Between Subjects Effects

Source DF Type III SS Mean Square

location 1 3.802500 3.802500
variety 2 1071.387222 535.693611
location*variety 2 841.031667 420.515833
Error 6 348.941667 58.156944

Source F Value Pr > F

location 0.07 0.8067
variety 9.21 0.0148
location*variety 7.23 0.0252
453
 Example: Peanuts(SAS Code)

Repeated Measures Analysis of Variance

Univariate Tests of Hypotheses for Within Subject Effects

Source DF Type III SS Mean Square

traits 2 126097.2156 63048.6078
traits*location 2 231.4867 115.7433
traits*variety 4 497.8444 124.4611
traits*location*variety 4 230.7167 57.6792
Error(traits) 12 202.2033 16.8503

454
 Example: Peanuts(SAS Code)

Adj Pr > F
Source F Value Pr > F G - G H - F
traits 3741.70 <.0001 <.0001 <.0001
traits*location 6.87 0.0103 0.0337 0.0103
traits*variety 7.39 0.0031 0.0188 0.0031
traits*location*variety 3.42 0.0436 0.0919 0.0436

Greenhouse-Geisser Epsilon 0.5583

Huynh-Feldt Epsilon 1.1672

455