Contrast

Contrasts
• Many multiple comparison methods use the
idea of a contrast.
• Considering the plasma etching experiment as
we studied previously, the null hypothesis and
the alternative hypothesis are:

H 0 : 1 =  2 = 3 =  4
vs.
H1 : not all i are equal
Figure: A single-wafer plasma etching tool.
Table: Etch Rate Data (in Å/min) from the Plasma Etching
Experiment)
We may state our Null hypothesis differently
such as:

H 0 : 3 =  4
H1 : 3   4
or

H 0 : 1 +  2 = 3 +  4
H1 : 1 +  2  3 +  4
The above two hypothesis testing are dealing with
“Contrasts”. In general, a “contrast” is a linear
combination of parameters of the following form:
k
 ci i
i =1
where the contrast constants c1, c2, c3, …ck sum to zero,
k
i.e.,
 ci = 0
i =1
The hypothesis are then in the form of
k
H 0 :  ci i = 0
i =1
k
H1 :  ci i  0
i =1
 Orthogonal Contrasts
A useful special case of the contrasts is that of
orthogonal contrasts. Two contrasts with
coefficients (ci) and (di) are orthogonal if
k
 ci di = 0 (balanced)
i =1
k
 ni ci di = 0
i =1 (unbalanced)
 Orthogonal Contrasts
For example, if there are k=3 levels (treatments) for
a factor, then the following are two possible
orthogonal contrasts for these 3 levels (treatments):

Treatment 1 Treatment 2 Treatment 3

(Level 1) (Level 2) (Level 3)
Coefficients -2 1 1
For
Orthogonal 0 -1 1
Contrasts
 Comparing All Contrasts in ANOVA

For a specific Contrast in the ANOVA table,

The associated Sum of Square can be calculated
as: k k
( ci yi. )
2 2
( ci yi. )
SSCi = i =1
= i =1
1 k 2 k

n i =1
ci n ci2
i =1

Notice that the difference is between whether treatment level total yi .

or treatment level mean y ( = yi . ) used in the equation.
i.
n
Comparing All Contrasts in ANOVA

If the sample sizes among different treatment

levels are not equal, the associated Contrast
Sum of Squares becomes:
k
k yi. 2
( ci yi. )2
( ci )
i =1 ni
SSC = k 2 =
i =1
k 2
i
ci ci
n n
i =1 i i =1 i
 Example: Plasma Etching

Hypothesis Contrast
H 0 : 1 =  2 C1 = y1. − y2.
H 0 : 1 +  2 = 3 +  4 C2 = y1. + y2. − y3. − y4.
H 0 : 3 =  4 C3 = y 3 . − y 4 .
The above 3 contrasts
are Orthogonal
 (−36.2) 2
C1=+1(551.2)-1(587.4)=-36.2 SSC1 = = 3276.10
1
( 2)
5
C2=+1(551.2)+1(587.4) (−193.8) 2
-1(625.4)-1(707.0)=-193.8 SSC2 = = 46,948.05
1
(4)
5
C3=+1(625.4)-1(707.0)=-81.6 (−81.6) 2
SSC3 = = 16,646.40
1
(2)
5
We can conclude from the P-Values that there are significant
differences in mean etch rates between levels 1 and 2, and
between levels 3 and 4 of the power setting, and the average
of levels 1 and 2 does differ significantly from the average
of Levels 3 and 4.
More discussion about contrasts
What ANOVA and F-Test Don’t Tell…
The ANOVA and F test as discussed were
limited to testing whether or not an overall
significant difference existed among the sample
means. We are often more interested in
specific comparisons or contrasts (the two
terms will be used interchangeably) between
two or more sample means than we are in an
overall test of significance. Just now, we
examined the procedures that can be used to
test hypotheses about specific differences
between two or more sample means when doing
an analysis of variance – CONTRASTS!
Linear Contrasts and Multiple Comparisons
If we reject H0 of no differences in treatment means
and in favor of H1, we conclude that at least one of the k
population means differs from the other k-1.
The Question is:
Which means differ from each other?
Multiple comparison procedures have been developed to help
determine which means are significantly different from each other.
Many different approaches - not all produce the same result.
Data dredging and data snooping - analyzing only those
comparisons which look interesting after looking at the data.

Problems with the confidence assumed for the comparisons.

1-a for a particular pre-specified comparison?

1-a for all unplanned comparisons as a group?
One procedure for making individual comparisons among
sample means is called the “method of planned (in
advance) orthogonal (independent) comparisons” or
“Linear Contrast and Multiple Comparison”. The
method of planned orthogonal comparisons replaces the
overall F test (based on dividing MSbt.-treatment by
MSwithin-treatment) with a series of up to k-1 individual
F tests, each with one degree of freedom for the
numerator. They also use MSe (or MSwithin-treatment) for the
denominator. Thus, instead just one F test with k-1
degrees of freedom for the denominator, planned
orthogonal comparisons makes possible (k-1) F tests
each with one degree of freedom for the numerator.
Example

ˆ  y2 + y3 
C1 = y1 −   Compare average of treatments 2 & 3 to treatment 1.
 2 
Contrast treatments 2 and 3.
Cˆ 2 = y2 − y3
c1 = 1
1
c2 = 01

1 Orthogonal
c1 = − c2 = 1
2
2 2

Cˆ 1= y1 − y2
1
c1 = − c2 = −1
3
2 3

Cˆ 2 = y1 − y3
c1 = 1 c2 = 1
Non-orthogonal 1 1

c = −1
Contrast treatment 2 to treatment 1. 1 2
c2 = 0 2
Contrast treatment 3 to treatment 1.
c1 = 0
3
c2 = −1 3
• To use this method, all your tests of significance must be
planned before looking at your data, and the tests of
significance must be independent of each other - which is
what limits it to (k-1) tests of significance.
• Suppose we are interested in the effects of six different
percentages of carbon contents in a new alloy on the
associated hardness, we have (6-1)=5 degrees of freedom
which means we can perform up to five independent tests.
• To figure out which comparisons we can perform, we must
first define “comparison” (or contrast). A comparison, Ci , is
defined as
C = c  + c  + ... + c 
i 1 1 2 2 k k
where k
 ci = 0
i =1
• For example, a contrast (comparison) between the first and second
mean would be a valid comparison by this formula because coefficient
c1 would equal a "+1" and coefficient c2 would equal a "-1" and all the
other coefficients would equal zero--thus all the ci would add up to zero.
Such a comparison would look like the following:

(+1)1 + (−1)2 + (0)3 + (0)4 + (0) 5 + (0)6

• A comparison between the first two means and the third and fourth
means would look like the following:
(+1)1 + (+1)2 + (−1)3 + (−1)4 + (0)5 + (0)6
The coefficients here could also have been: 1/2, 1/2, -1/2, -1/2, 0, and 0.
• Another comparison between the first four means and the last two
means would look like the following:
(+1)1 + (+1)2 + (+1)3 + (+1)4 + (−2)5 + (−2)6
The values for the coefficients could also have been 1/4, 1/4, 1/4, 1/4,
-1/2, & -1/2 instead of 1, 1, 1, 1, -2, &-2 because these values would also
have added up to zero.
• Two comparisons C1 and C2 are independent (mutually
orthogonal) if the sum of the products their respective
coefficients equal zero; in other words, if .
k
 c1 c2 i i
=0
i =1
For example, the two contrasts (comparisons)

C1 = (+1)1 + (−1)2 + (0)3 + (0)4 + (0) 5 + (0)6

and

C2 = (+1)1 + (+1)2 + (−1)3 + (−1)4 + (0)5 + (0)6

are independent comparisons (orthogonal) because

(+1)(+1) + (-1)(+1) + (0)(-1) + (0)(-1) + (0)(0)+ (0)(0) = 0
Contrasts (comparisons) C1 and C3 ; and C2 and
C3 are also independent (orthogonal) where
C3 = (+1)1 + (+1)2 + (+1)3 + (+1)4 + (−2)5 + (−2)6
because in each case . k
 c1i c3i =0
i =1
k
 c2i c3i =0
i =1

A set of three or more contrasts are said to be mutually

orthogonal (or mutually independent) if all pairs of linear
contrasts are orthogonal.
 Two more independent comparisons (contrasts)
are possible. The following two comparisons
are independent of the three we have already
identified:

C4 = (0)1 + (0)2 + (1)3 + (−1)4 + (0)5 + (0)6

C5 = (0)1 + (0)2 + (0)3 + (0)4 + (+1)5 + (−1)6

 Importance of Mutual Orthogonality

Assume k treatment groups, each group having n individuals (units).

• k-1 mutually orthogonal contrasts can be formed from the k

means. (Remember k-1 degrees of freedom.)
• Treatment sums of squares (SSbt-treatment) can be computed as
the sum of the sums of squares associated with the k-1
orthogonal contrasts. (i.e. the treatment sums of squares can
be partitioned into k-1 parts associated with k-1 mutually
orthogonal contrasts).

contrasts orthogonal  SSC1 +  + SSCk −1 = SSbt −treatment

k-1 independent pieces of information about the

variability in the treatment means.
Once we have identified our comparisons and determined that
they are independent (orthogonal), our next step is to compute
the part of SSbt-treatment that belongs to each contrast
(comparison). (Note that ultimately the sum of squares for our
five orthogonal contrasts (comparisons) should add up to
SSbt-treatment)
As we did last lecture,
k k
( ci yi. ) 2 ( ci yi. ) 2
when sample size are equal
SSCi = i =1
k
= i =1
k
1 among different treatments

n i =1
ci
2
n ci2
i =1
k
k yi. 2
( ci yi. ) ( ci )2
when sample size are not equal
i =1 ni
SSC = k 2 =
i =1
among different treatments
i
ci k
ci2
n n
i =1 i i =1 i
 For the 5 orthogonal contrast, the associated SSCi are:

[( +1) y1. + (−1) y2. + (0) y3. + (0) y4. + (0) y5. + (0) y6. ]2 [ y1. − y2. ]2
SSC1 = =
1 1
[( +1) + (−1) + (0) + (0) + (0) + (0) ]
2 2 2 2 2 2
[2]
n n
[( +1) y1. + (+1) y2. + (−1) y3. + (−1) y4. + (0) y5. + (0) y6. ]2
SSC2 =
1
[( +1) 2 + (+1) 2 + (−1) 2 + (−1) 2 + (0) 2 + (0) 2 ]
n
[ y1. + y2. − y3. − y4. ]2
=
1
[ 4]
n
[( +1) y1. + (+1) y2. + (+1) y3. + (+1) y4. + (−2) y5. + (−2) y6. ]2
SSC3 =
1
[( +1) 2 + (+1) 2 + (+1) 2 + (+1) 2 + (−2) 2 + (−2) 2 ]
n
[ y1. + y2. + y3. + y4. − 2 y5. − 2 y6. ]2
=
1
[12]
n
 [(0) y1. + (0) y2. + (+1) y3. + (−1) y4. + (0) y5. + (0) y6. ]2 [ y3. − y4. ]2
SSC4 = =
1 1
[(0) + (0) + (+1) + (−1) + (0) + (0) ]
2 2 2 2 2 2
[2]
n n

[(0) y1. + (0) y2. + (0) y3. + (0) y4. + (+1) y5. + (−1) y6. ]2 [ y5. − y6. ]2
SSC5 = =
1 1
[(0) + (0) + (0) + (0) + (+1) + (−1) ]
2 2 2 2 2 2
[2]
n n
 ANOVA FOR CONTRASTS
Source SS df MS F Fcv

Between k-1=5

C1 from previous slide 1 SSC1/1 MSC1/ Mse F1,k(n-1)

C2 from previous slide 1 SSC2/1 MSC2/ Mse F1,k(n-1)

C3 from previous slide 1 SSC3/1 MSC3/ Mse F1,k(n-1)

C4 from previous slide 1 SSC4/1 MSC4/ Mse F1,k(n-1)

C5 from previous slide 1 SSC5/1 MSC5/ MSe F1,k(n-1)

Error SSe/
k(n-1)
(Within) k(n-1)

Total kn-1
Additional Example of Linear Contrasts

Objective: Test the wear quality of a new paint.

Treatments: Weather and wood combinations.

Treatment Code Combination

A 1 hardwood, dry climate
B 2 hardwood, wet climate
C 3 softwood, dry climate
D 4 softwood, wet climate
Questions To Be Answered by the experiment:
Q1: Is the average life on hardwood the same as average life
on softwood?
Q2: Is the average life in dry climate the same as average life
in wet climate?
Q3: Does the difference in paint life between wet and dry
climates depend upon whether the wood is hard or soft?
 Treatment A B C D
Mean
(in years) 13 14 20 21
MSe = 5 Q1
k= 4
ni 3 3 3 3
Population (n-1)k 8
parameter 1 2 3 4

Q1: Is the average life on hardwood the same as average life on softwood?
1 + 2 3 +  4  1 +  2   3 +  4 
1
H0 : = OR  − 2  = 0
2 2  2   
1 1 1 1
C1 = ( ) 1 + ( )  2 + (− ) 3 + (− )  4
Comparison: 2 2 2 2
ˆ 1 1 1 1
Estimated Contrast C1 = ( ) y1 + ( ) y2 + (− ) y3 + (− ) y4
2 2 2 2
Test H0: C1 = 0 versus H1: C1  0
MS C1
Test Statistic:F =
MS e
Rejection Region: Reject H0 if F  F1,( n −1) k ,a
 k
1
( c1i yi ) 2 1 1 1
[( )13 + ( )14 + (− )20 + (− )21]2
MS C = 1 k 2 = 2 2 2 2 = 147
1
c1i 1 2 1 2 1 2 1 2
( ) ( ) ( ) ( − ) ( − )
i =1 ni [ 2 + 2 + 2 + 2
3 3 3 3
MS C
F= 1
= 147 / 5 = 29.4
MS e
F1,8, 0.05 = 5.32

Conclusion: Since F=29.4 > 5.32 we reject H0 and conclude

that there is a significant difference in average life on hard
versus soft woods.
 Treatment
Mean
(in years)
A

13
B

14
C

20
D

21
Q2
ni 3 3 3 3 MSe= 5
Population k= 4
parameter 1 2 3 4 (n-1)k 8

Q2: Is the average life in dry climate the same as average life in wet climate?
1 + 3 2 + 4  1 + 3    2 +  4 
H0 : = OR  − =0
2 2  2   2 
1 1 1 1
Comparison: C2 = ( 2 ) 1 + (− 2 )  2 + ( 2 ) 3 + (− 2 )  4
1 1 1 1
Estimated Contrast Cˆ 2 = ( ) y1 + (− ) y2 + ( ) y3 + (− ) y4
2 2 2 2
Test H0: C2 = 0 versus H1: C2  0
MS C
Test Statistic: F= 2

MS e
Rejection Region: Reject H0 if F  F1,( n −1) k ,a
 k
(  c 2 yi ) 21 1 1 1
[( )13 + (− )14 + ( )20 + (− )21]2
MS C = 1k 2 = 2 2 2 2 =3
2
c2 i 1 2 1 2 1 2 1 2
( ) ( ) (− ) ( ) (− )
i =1 ni [ 2 + 2 + 2 + 2
3 3 3 3

MS C
F= 2
= 3 / 5 = 0.6
MS e
F1,8, 0.05 = 5.32
Conclusion: Since F=0.6 < 5.32 we do not reject H0 and
conclude that there is not a significant difference in average life
in wet versus dry climates.
 Treatment
Mean
A B C D Q3
(in years) 13 14 20 21
ni 3 3 3 3 MSe= 5
Population t= 4
parameter 1 2 3 4 (n-1)k= 8

Q3: Does the difference in paint life between wet and dry climates depend
upon whether the wood is hard or soft?
H30 : 1 − 2 = 3 − 4 OR (1 − 2 ) − (3 − 4 ) = 0

Comparison: C3 = (1) 1 + (−1)  2 + (−1) 3 + (1)  4

Estimated Contrast Cˆ 3 = (1) y1 + (−1) y2 + (−1) y3 + (1) y4
Test H0: C3 = 0 versus H1: C3  0
MS C
Test Statistic: F= 3

MS e
Rejection Region: Reject H0 if F  F1,( n −1) k ,a
 k
( c3 yi ) 2
[(1)13 + ( −1)14 + ( −1) 20 + (1) 21]2
MS C = 1k 2 = =0
3
c3i (1) (−1) (−1) (1)
2 2 2 2
( ) [ + + +
i =1 ni 3 3 3 3

MS C
F= 3
= 0 / 5 = 0.0
MS e
F1,8, 0.05 = 5.32

Conclusion: Since F=0 < 5.32 we do not reject H0 and

conclude that the difference between average paint life between
wet and dry climates does not depend on wood type. Likewise,
the difference between average paint life for the wood types
does not depend on climate type (i.e. there is no interaction).
 Mutual Orthogonality
Contrast
l1Cl12 C2 = 1
4
− 14 − 14 + 14 =0
C1 1/2 1/2 -1/2 -1/2
C2 1/2 -1/2 1/2 -1/2
l1Cl13 C3 = 1
2
− 12 + 12 − 214 =0
C3 1 -1 -1 1 l2Cl23 C3 = 1
2
+ 12 − 12 − 12 =0
The three are mutually orthogonal.
SSC1 = MSC1 = 147
The three mutually orthogonal contrasts
SSC2 = MSC2 = 3
add up to the Treatment Sums of
SSC3 = MSC3 = 0
Squares.
Treatment SS = 150

Total Error SS = dferror x MSe = 8 x 5 = 40

Why?
 It is not necessary to perform all k-1 tests of
significance. Two or more of the sums of squares
could be lumped together as a sums of squares
residual. For instance, in the earlier example
with 5 contrasts, SSC4 and SSC5 could be
lumped together into a SSres and its degrees of
freedom would be equal to 2. SSres/2 would give
us MSres with 2 degrees of freedom. If we lumped
all the five sums of squares comparisons
together we would have SSbt-treatment,
i.e.,
SSbt-treatment = SSC1+SSC2 + SSC3 + SSC4 + SSC5.
Planned orthogonal comparisons control the error rate,
or probability of committing a type one error (calling a null
hypothesis false when it is true), on a per-comparison
basis. In other words, the level of alpha (usually .01 or
.05), or probability of committing a type one error, is set
individually for each comparison irrespective of however
many tests of significance are performed. This means that
the degree of improbability considered sufficient to reject
the null hypothesis is considered for each comparison as
if it were the only one being performed. When we perform
k-1 independent tests of significance the probability that
we have committed at least one type one error is equal
to k −1
1 − (1 − a )
where alpha, is the level of significance or probability of
committing a type one error given that the null hypothesis
is true.
 Types of Error Rates
Number of Type I Experimentwise
Comparison-wise Error Rate - the comparsons Error Rate
1 0.05
Error Rate
0.050
probability of making a Type I error in 2 0.05 0.098
3 0.05 0.143
the comparison of two means. (what 4 0.05 0.185
we have been discussing for all tests 5 0.05 0.226
6 0.05 0.265
up to this point). 7 0.05 0.302
8 0.05 0.337
9 0.05 0.370
10 0.05 0.401
11 0.05 0.431
Experiment-wise Error Rate - the 12 0.05 0.460
13 0.05 0.487
probability of observing an 14 0.05 0.512
experiment in which one or more of 15 0.05 0.537
16 0.05 0.560
the pair-wise comparisons are 17 0.05 0.582
18 0.05 0.603
incorrectly declared significantly 19 0.05 0.623
different. 20 0.05 0.642
In our example, had we used an alpha level of .05 (5%)
for each comparison, our overall probability of
committed at least one Type-I error, if all five of our null
hypotheses were true, would be about .226 (23%)
instead of .05 (5%).
[1-(1-0.05)5]=22.6%

The value of .226 is called an experiment-wise error

rate as compared with a comparison-wise error rate.
It determines the error rate for the experiment as a
whole, rather than for each individual comparison.
Later, we will consider a method for making any or all
comparisons whether they are planned or not. It is
strictly a per-experiment or experiment-wise method for
controlling error rate.
 Error Rates: Problems
Number of Type I Experimentwise
Suppose we make k-1 mutually orthogonal comparsons Error Rate Error Rate
1 0.05 0.050
(independent) comparisons, each with Type I 2 0.05 0.098
comparison-wise error rate of a. The 3 0.05 0.143
4 0.05 0.185
experiment-wise error rate, e, is then: 5 0.05 0.226
6 0.05 0.265
7 0.05 0.302

e = 1 − (1 − a ) k −1 8 0.05 0.337
9 0.05 0.370
10 0.05 0.401
11 0.05 0.431
12 0.05 0.460
(If the comparisons are not orthogonal, then the 13
14
0.05
0.05
0.487
0.512
15 0.05 0.537
experiment-wise error rate is smaller.) 16 0.05 0.560
17 0.05 0.582
Solution (by Bonferroni): set e=0.05 and solve for 18 0.05 0.603

a. But there’s a problem… 19

20
0.05
0.05
0.623
0.642

E.g. if k-1=8, we get a=0.0064!