Comparisons of Several Multivariate Means

(two way MANOVA)

Pertemuan8-APG-
Outline
• Introduction two way MANOVA
• Data Structure
• Illustrations: Interaction & No Interaction in Univariate Data
• Two way MANOVA with interaction:
• Two way MANOVA Table
• Two way MANOVA Inferences
• Simultaneous Confidence Interval
• Application in R

2 Comparison of Several Multivariate Means

Introduction
The two-way multivariate analysis of variance (two-way MANOVA) is often considered
as an extension of the two-way ANOVA for situations where there is two or more
dependent variables.
The primary purpose of the two-way MANOVA is to understand if there is an
interaction between the two independent variables on the two or more dependent
variables.

Dependent/Response Variable Independent Variable/factor

• Two factor
• Two or more • Categorical
• Interval/ratio variable
(numeric) • Fixed effect

3 Comparison of Several Multivariate Means

Illustration of Data Structure if No Interaction
Two way ANOVA Two way MANOVA

Factor 1 Factor 1
level 1 2 … 𝑔 level 1 2 … 𝑔
1 𝑥11 𝑥21 𝑥𝑔1 𝑥ҧ ∙1
Factor 2

2 𝑥12 𝑥22 𝑥𝑔2 𝑥ҧ ∙2 𝑋1 𝑋2 𝑋1 𝑋2 𝑋1 𝑋2

⋮ ⋮ ⋮ ⋮
𝑥ҧ∙𝑘
1 𝑥111 𝑥112 𝑥211 𝑥212 𝑥𝑔11 𝑥𝑔12 𝐱ത ∙𝟏
𝑏 𝑥1𝑏 𝑥2𝑏 𝑥𝑔b 𝑥ҧ∙𝑏
2 𝑥121 𝑥122 𝑥221 𝑥222 𝑥𝑔21 𝑥𝑔22 𝐱ത ∙𝟐

Factor 2
𝑥ҧ1∙ 𝑥ҧ2∙ 𝑥𝑔∙
ҧ 𝐱ത ∙𝒌
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

𝑥ҧ𝑙∙ 𝑏 𝑥1b1 𝑥1b2 𝑥2b1 𝑥2b2 𝑥𝑔b1 𝑥𝑔b2 𝐱ത ∙𝒃

𝐱ത 1∙ 𝐱ത 2∙ 𝐱ത𝑔∙

𝐱ത 𝒍∙
If there are any replications, so we can analyze such data with interaction analysis

4 Comparison of Several Multivariate Means

 Illustrations: Interaction & No Interaction in
Univariate

Interaction No Interaction

5 Comparison of Several Multivariate Means

 Factor 1
Two way level 1 2 … 𝑔
MANOVA
with 𝑋1 𝑋2 𝑋1 𝑋2 𝑋1 𝑋2

interaction 1 1 𝑥1111 𝑥1121 𝑥2111 𝑥2121 𝑥𝑔111 𝑥𝑔121

⋮ 𝐱ത 11 𝐱ത 21 𝐱ത𝑔1 𝐱ത ∙𝟏
𝑛 𝑥111𝑛 𝑥112𝑛 𝑥211𝑛 𝑥212𝑛 𝑥𝑔11𝑛 𝑥𝑔12𝑛
𝑿𝑙𝑘𝑟 ~𝑁𝑝 𝝁𝑙𝑘 , 𝜮
2 1 𝑥1211 𝑥1221 𝑥2211 𝑥2221 𝑥𝑔211 𝑥𝑔221

Factor 2 𝐱ത𝑔2 𝐱ത ∙𝟐
⋮ 𝐱ത 12 𝐱ത 22
The assumptions are
◮ Multivariate normality. 𝑛 𝑥121𝑛 𝑥122𝑛 𝑥221𝑛 𝑥222𝑛 𝑥𝑔21𝑛 𝑥𝑔22𝑛
◮ Equality of covariance ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
matrix ’s.
𝑏 1 𝑥1b11 𝑥1b21 𝑥2b11 𝑥2b21 𝑥𝑔b11 𝑥𝑔b21
◮ Independence between
and within groups. ⋮ 𝐱ത 2𝑏 𝐱ത𝑔𝑏 𝐱ത ∙𝒃
𝐱ത1𝑏
𝑛 𝑥1b1𝑛 𝑥1b2𝑛 𝑥2b1𝑛 𝑥2b2𝑛 𝑥𝑔b1𝑛 𝑥𝑔b2𝑛

𝐱ത1∙ 𝐱ത 2∙ 𝐱ത𝑔∙
6 Comparison of Several Multivariate Means
Two way MANOVA with interaction
𝑿𝑙𝑘𝑟 ~𝑁𝑝 𝝁𝑙𝑘 , 𝜮 ; 𝑙 = 1,2, … , 𝑔; 𝑘 = 1, … , 𝑏; 𝑟 = 1, … , 𝑛

The two-way multivariate analysis of variance (MANOVA) has the form:

𝑿𝑙𝑘𝑟 = 𝝁 + 𝝉𝑙 + 𝜷𝑘 + 𝜸𝑙𝑘 + 𝒆𝑙𝑘𝑟
𝑔 𝑔
where σ𝑙=1 𝝉𝑙 = σ𝑘=1 𝜷𝑘 = σ𝑙=1 𝜸𝑙𝑘 = σ𝑏𝑘=1 𝜸𝑙𝑘 = 𝟎
𝑏

The sample estimates of the model parameters are:

𝐱 𝑙𝑘𝑟 = 𝐱ത + 𝐱ത 𝒍∙ − 𝐱ത + 𝐱ത ∙𝒌 − 𝐱ത + 𝐱ത 𝒍𝒌 − 𝐱ത 𝒍∙ − 𝐱ത ∙𝒌 + 𝐱ത + 𝐱 𝑙𝑘𝑟 − 𝐱ത 𝑙𝑘
=𝝁 ෝ + 𝝉ො 𝒍 +𝜷 ෡𝑘 +𝜸ෝ𝑙𝑘 + 𝒆ො 𝑙𝑘𝑟

𝐱 𝑙𝑘𝑟 = observations/ responses consist of 𝑝 measurement

𝝁ෝ = 𝐱ത = overall sample mean is an estimate of 𝜇
𝝉ො 𝒍 = 𝐱ത 𝒍. − 𝐱ത = estimated main effect of factor 1 and 𝐱ത 𝒍. is the average of the observation
vectors at the 𝑙th level of factor 1.
𝜷෡ 𝑘 = 𝐱ത ∙𝒌 − 𝐱ത = estimated main effect of factor 2 and 𝐱ത .𝒌 is the average of the observation
vectors at the 𝑘th level of factor 2.
𝜸ෝ𝑙𝑘 = 𝐱ത 𝒍𝒌 − 𝐱ത 𝒍∙ − 𝐱ത ∙𝒌 + 𝐱ത = estimated interaction effect between factor 1 and factor 2, 𝐱ത 𝒍𝒌 is
the average of observation vectors at the 𝑙th level of factor 1 and 𝑘th level of factor 2.
𝒆ො 𝑙𝑘𝑟 = 𝐱 𝑙𝑘𝑟 − 𝐱ത 𝑙𝑘 = residual
7 Comparison of Several Multivariate Means
Two Way MANOVA Table

8 Comparison of Several Multivariate Means

Two Way MANOVA Inference
Test for interaction effect:
𝐻0 : 𝜸11 = 𝜸12 = ⋯ = 𝜸𝑔𝑏 = 𝟎 (no interaction effect) vs 𝐻1 : at least one 𝜸𝑙𝑘 ≠ 𝟎
𝑆𝑆𝑃𝑟𝑒𝑠
Λ∗=
𝑆𝑆𝑃𝑖𝑛𝑡 + 𝑆𝑆𝑃𝑟𝑒𝑠
Reject 𝐻0 at the 𝛼 level if:
𝑝+1− 𝑔−1 𝑏−1
− 𝑔𝑏 𝑛 − 1 − ln Λ ∗ > 𝜒 2 𝑔−1 𝑏−1 𝑝 ;𝛼
2

• If 𝐻0 for the interaction is rejected, then you don’t so tests for main effects because the Factor
main effects do not have a clear interpretation.
• Jhonson & Winchern recommend doing p-univariate 2-way ANOVAs to find out “where” the
interactions exist.
• For variables with no interaction, you can interpret main effects.
• Note: In some cases you can go on to test main effects even though the interaction is significant
(we’ll do this in our example).

9 Comparison of Several Multivariate Means

Test for Main Effect

Test for factor 1 main effect:

𝐻0 : 𝝉𝟏 = 𝝉𝟐 = ⋯ = 𝝉𝒈 = 𝟎 (no factor 1 effect) vs 𝐻1 : at least one 𝝉𝑙 ≠ 𝟎
𝑆𝑆𝑃𝑟𝑒𝑠
Λ∗=
𝑆𝑆𝑃fact1 + 𝑆𝑆𝑃𝑟𝑒𝑠
Reject 𝐻0 at the 𝛼 level if:
𝑝+1− 𝑔−1
− 𝑔𝑏 𝑛 − 1 − ln Λ ∗ > 𝜒 2 𝑔−1 𝑝 ;𝛼
2

Test for factor 2 main effect:

𝐻0 : 𝜷𝟏 = 𝜷𝟐 = ⋯ = 𝜷𝒃 = 𝟎 (no factor 2 effect) vs 𝐻1 : at least one𝜷𝒌 ≠ 𝟎
𝑆𝑆𝑃𝑟𝑒𝑠
Λ∗=
𝑆𝑆𝑃fact2 + 𝑆𝑆𝑃𝑟𝑒𝑠
Reject 𝐻0 at the 𝛼 level if:
𝑝+1− 𝑏−1
− 𝑔𝑏 𝑛 − 1 − ln Λ ∗ > 𝜒 2 𝑏−1 𝑝 ;𝛼
2

10 Comparison of Several Multivariate Means

Simultaneous Confidence Intervals.
• When interaction effects are negligible, we may concentrate on contrasts in factor 1 and
factor 2 main effect.
• If a null hypothesis (test for main effect) is rejected, you should perform additional
analyses to figure out where the effects are.
• The Bonferroni approach applies to the components of the differences 𝝉𝒍 − 𝝉𝒎 of the
factor 1 effects and the components of 𝜷𝒌 − 𝜷𝒒 of the factor 2 effects.

The Bonferroni 100 1 − 𝛼 % simul. confidence interval for 𝜏𝑙𝑖 − 𝜏𝑚𝑖 , 𝑖 = 1, … , 𝑝 variable
and 𝑚 < 𝑙 = 1, … , 𝑔 :
𝐸𝑖𝑖 2
𝑥ҧ𝑙𝑖∙ − 𝑥ҧ𝑚𝑖∙ ± 𝑡 𝛼
𝑣;
𝑝𝑔 𝑔−1 𝑣 𝑏𝑛
Where 𝑣 = 𝑔𝑏 𝑛 − 1 , 𝐸𝑖𝑖 is diagonal element of 𝑬 = 𝑆𝑆𝑃𝑟𝑒𝑠 and 𝑥ҧ𝑙𝑖∙ − 𝑥ҧ𝑚𝑖∙ is the 𝑖th
component of 𝐱ത 𝑙∙ − 𝐱ത 𝑚∙

11 Comparison of Several Multivariate Means

The Bonferroni 100 1 − 𝛼 % simul. confidence interval for 𝛽𝑘𝑖 − 𝛽𝑞𝑖 , 𝑖 = 1, … , 𝑝 variable
and 𝑞 < 𝑘 = 1, … , 𝑏 :
𝐸𝑖𝑖 2
𝑥ҧ∙𝑘𝑖 − 𝑥ҧ∙𝑞𝑖 ± 𝑡 𝛼
𝑣;
𝑝𝑏 𝑏−1 𝑣 𝑔𝑛
Where 𝑣 = 𝑔𝑏 𝑛 − 1 , 𝐸𝑖𝑖 is diagonal element of 𝑬 = 𝑆𝑆𝑃𝑟𝑒𝑠 and𝑥ҧ∙𝑘𝑖 − 𝑥ҧ∙𝑞𝑖 is the 𝑖th
component of 𝐱ത ∙𝑘 − 𝐱ത ∙𝑞

12 Comparison of Several Multivariate Means

Example 6.11 (Jhonson & Wichern, 2002)
The optimum conditions for
extruding plastic film have
been examined using a
technique called
Evolutionary Operation. In
the course of the study
that was done, three
responses-X1 = tear
resistance, X2 = gloss, and
X3 = opacity-were
measured at two levels of
the factors, rate of
extrusion and amount of
an additive. The
measurements were
repeated 𝑛 = 5 times at
each combination of the
factor levels.
13 Comparison of Several Multivariate Means
 dataFilm<-read.csv("D://STIS/05_APG terapan D3/Data Contoh/FILM.csv"
, sep=";")
X <- as.matrix(dataFilm[,c("X1","X2","X3")])
mod <- lm(X ~ Factor1*Factor2,data=dataFilm)
library(car)

## Warning: package 'car' was built under R version 4.0.4

## Loading required package: carData

Manova(mod, test="Wilks")

##
## Type II MANOVA Tests: Wilks test statistic
## Df test stat approx F num Df den Df Pr(>F)
## Factor1 1 0.38186 7.5543 3 14 0.003034 **
## Factor2 1 0.52303 4.2556 3 14 0.024745 *
## Factor1:Factor2 1 0.77711 1.3385 3 14 0.301782
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Wilk’s LambdaΛ ∗

14 Comparison of Several Multivariate Means