Anova and the

Design of Experiments

.Why and How to use ANOVA in Marketing Research?

• What is the Difference between one way ANOVA, N-way

ANOVA, MANOVA and ANCOVA
• Analysis of variance (ANOVA) is used as a test of means for
two or more populations. The null hypothesis, typically, is that
all means are equal.
• Analysis of variance must have a dependent variable that is
metric (measured using an interval or ratio scale).
• There must also be one or more independent variables that are
all categorical (nonmetric). Categorical independent variables
are also called factors.
• Design
• Completely Randomised Design in a One-Way ANOVA (Single
Factor)
• Randomized Block Design (Single Blocking Factor)
• Factorial Design with 2 or more Factors.
 One-way Analysis of Variance
Application in MR

• Do the various segments differ in terms of their volume

of product consumption?
• Do the brand evaluations of groups exposed to different
commercials vary?
• What is the effect of consumers' familiarity with the store
(measured as high, medium, and low) on preference for
the store?
Relationship Amongst Test, Analysis of
Variance, Analysis of Covariance, &
Regression Metric Dependent Variable

One Independent
Variable One or More
Independent Variables

Categorical: Categorical
Binary Interval
Factorial and Interval

Analysis of Analysis of
t Test Variance Covariance Regression

More than
One Factor One Factor

One-Way Analysis N-Way Analysis

of Variance of Variance
Conducting
Fig. 16.2
One-way ANOVA
Identify the Dependent and Independent Variables

Decompose the Total Variation

Measure the Effects

Test the Significance

Interpret the Results

 Conducting One-way Analysis of
Variance
Decompose the Total Variation
The total variation in Y, denoted by SSy, can be decomposed
into two components:
 
SSy = SSbetween + SSwithin
 
where the subscripts between and within refer to the
categories of X. SSbetween is the variation in Y related to the
variation in the means of the categories of X. For this reason,
SSbetween is also denoted as SSx. SSwithin is the variation in Y
related to the variation within each category of X. SSwithin is
not accounted for by X. Therefore it is referred to as SSerror.
 Decomposition of the Total Variation:
Table 16.1 One-way ANOVA
Independent Variable X
Total
Categories Sample
Within X1 X2 X3 … Xc
Category Total
Y1 Y1 Y1 Y1 Y1 Variation
Variation
Y2 Y2 Y2 Y2 Y2 =SSy
=SSwithin
: :
: :
Yn Yn Yn Yn YN
Category
Mean Y1 Y2 Y3 Yc Y

Between Category Variation = SSbetween

 Conducting One-way Analysis of
Variance
In analysis of variance, we estimate two measures of variation:
within groups (SSwithin) and between groups (SSbetween). Thus,
by comparing the Y variance estimates based on between-
group and within-group variation, we can test the null
hypothesis.

Measure the Effects

The strength of the effects of X on Y are measured as follows:
 
2
= SSx/SSy = (SSy - SSerror)/SSy
 

The value of 2 varies between 0 and 1.


 Conducting One-way Analysis of
Variance
Test Significance
In one-way analysis of variance, the interest lies in testing the null hypothesis that the
category means are equal in the population.
 
H0: µ1 = µ2 = µ3 = ........... = µc
 
Under the null hypothesis, SSx and SSerror come from the same source of variation. In
other words, the estimate of the population variance of Y,

= SS
2 x/(c - 1)
S
= Mean
y square due to X
= MSx
or

= SSerror/(N - c)
= Mean
2 square due to error
S
= MS
y error
 Conducting One-way Analysis of
Variance
Test Significance
The null hypothesis may be tested by the F statistic
based on the ratio between these two estimates:
  SS x /(c - 1) MS x
  F = =
SS error/(N - c) MS error

This statistic follows the F distribution, with (c - 1) and

(N - c) degrees of freedom (df).
 Conducting One-way Analysis of
Variance
Interpret the Results
• If the null hypothesis of equal category means is not
rejected, then the independent variable does not have a
significant effect on the dependent variable.
• On the other hand, if the null hypothesis is rejected, then
the effect of the independent variable is significant.
• A comparison of the category mean values will indicate
the nature of the effect of the independent variable.
 Illustrative Applications of One-way
Analysis of Variance
We illustrate the concepts discussed in this chapter using
the data presented in Table 16.2.

The department store is attempting to determine the effect

of in-store promotion (X) on sales (Y). For the purpose of
illustrating hand calculations, the data of Table 16.2 are
transformed in Table 16.3 to show the store sales (Yij) for
each level of promotion.
 
The null hypothesis is that the category means are equal:
H0: µ1 = µ2 = µ3.
Effect of Promotion and Clientele
Table 16.2
Store Num ber
on Sales
Coupon Level In-Store Prom otion Sales Clientel Rating
1 1.00 1.00 10.00 9.00
2 1.00 1.00 9.00 10.00
3 1.00 1.00 10.00 8.00
4 1.00 1.00 8.00 4.00
5 1.00 1.00 9.00 6.00
6 1.00 2.00 8.00 8.00
7 1.00 2.00 8.00 4.00
8 1.00 2.00 7.00 10.00
9 1.00 2.00 9.00 6.00
10 1.00 2.00 6.00 9.00
11 1.00 3.00 5.00 8.00
12 1.00 3.00 7.00 9.00
13 1.00 3.00 6.00 6.00
14 1.00 3.00 4.00 10.00
15 1.00 3.00 5.00 4.00
16 2.00 1.00 8.00 10.00
17 2.00 1.00 9.00 6.00
18 2.00 1.00 7.00 8.00
19 2.00 1.00 7.00 4.00
20 2.00 1.00 6.00 9.00
21 2.00 2.00 4.00 6.00
22 2.00 2.00 5.00 8.00
23 2.00 2.00 5.00 10.00
24 2.00 2.00 6.00 4.00
25 2.00 2.00 4.00 9.00
26 2.00 3.00 2.00 4.00
27 2.00 3.00 3.00 6.00
28 2.00 3.00 2.00 10.00
29 2.00 3.00 1.00 9.00
30 2.00 3.00 2.00 8.00
 Illustrative Applications of One-way
Analysis of Variance
It can be verified that
SSy = SSx + SSerror
as follows:
185.867 = 106.067 +79.80
The strength of the effects of X on Y are measured as follows:
2 = SSx/SSy
= 106.067/185.867
= 0.571
 
In other words, 57.1% of the variation in sales (Y) is accounted for by
in-store promotion (X), indicating a modest effect. The null
hypothesis may now be tested.
  SS x /(c - 1)
F= = MS X
  SS error/(N - c) MS error
106.067/(3-1)
F=
79.800/(30-3)
= 17.944
 Illustrative Applications of One-way
Analysis of Variance
• From Table 5 in the Statistical Appendix we see that for
2 and 27 degrees of freedom, the critical value of F is
3.35 for = 0.05
. Because the calculated value of F is
greater than the critical value, we reject the null
hypothesis.
• We now illustrate the analysis of variance procedure
using a computer program. The results of conducting the
same analysis by computer are presented in Table 16.4.
 One-Way ANOVA:
Effect of In-store Promotion on Store
Table 16.3
Sales
Source of Sum of df Mean F ratio F prob.
Variation squares square
Between groups 106.067 2 53.033 17.944 0.000
(Promotion)
Within groups 79.800 27 2.956
(Error)
TOTAL 185.867 29 6.409

Cell means

Level of Count Mean

Promotion
High (1) 10 8.300
Medium (2) 10 6.200
Low (3) 10 3.700
TOTAL 30 6.067
N-way Analysis of Variance
In marketing research, one is often concerned with the
effect of more than one factor simultaneously. For
example:

• How do advertising levels (high, medium, and low) interact

with price levels (high, medium, and low) to influence a
brand's sale?

• Do educational levels (less than high school, high school

graduate, some college, and college graduate) and age
(less than 35, 35-55, more than 55) affect consumption of
a brand?

• What is the effect of consumers' familiarity with a

department store (high, medium, and low) and store
image (positive, neutral, and negative) on preference for
the store?
 N-way Analysis of Variance
Consider the simple case of two factors X1 and X2 having categories c1 and c2.
The total variation in this case is partitioned as follows:
 
SStotal = SS due to X1 + SS due to X2 + SS due to interaction of X1 and X2 + SSwithin
 
or
 
SS y = SS x 1 + SS x 2 + SS x 1x 2 + SS error
 
 
The strength of the joint effect of two factors, called the overall effect, or multiple
2
, is measured as follows:
  
  Multiple 2 = (SS x 1 + SS x 2 + SS x 1x 2)/ SS y
N-way Analysis of Variance
The significance of the overall effect may be tested by an F test, as
follows

(SS x 1 + SS x 2 + SS x 1x 2)/dfn
F=
SS error/dfd
SS x 1,x 2,x 1x 2/ dfn
=
SS error/dfd
MS x 1,x 2,x 1x 2
=
MS error
where
 
dfn = degrees of freedom for the numerator
= (c1 - 1) + (c2 - 1) + (c1 - 1) (c2 - 1)
= c1c2 - 1
dfd = degrees of freedom for the denominator
= N - c1c2
MS = mean square
 N-way Analysis of Variance
If the overall effect is significant, the next step is to examine the
significance of the interaction effect. Under the null hypothesis of no
interaction, the appropriate F test is:
SS x 1x 2/dfn
F=
SS error/dfd
MS x 1x 2
=
MS error

where
 
dfn = (c1 - 1) (c2 - 1)
dfd = N - c 1c 2
 N-way Analysis of Variance
The significance of the main effect of each factor may be tested
as follows for X1:
SS x 1/dfn
F=
SS error/dfd
MS x 1
=
MS error

where

dfn = c1 - 1
dfd = N - c 1c 2
 Two-way
Table 16.4
Analysis of Variance
Source of Sum of Mean Sig. of
Variation squares df square F F 2
Main Effects
Promotion 106.067 2 53.033 54.862 0.000 0.557
Coupon 53.333 1 53.333 55.172 0.000 0.280
Combined 159.400 3 53.133 54.966 0.000
Two-way 3.267 2 1.633 1.690 0.226
interaction
Model 162.667 5 32.533 33.655 0.000
Residual (error) 23.200 24 0.967
TOTAL 185.867 29 6.409
 Two-way
Table 16.4 cont.
Analysis of Variance
Cell Means
Promotion Coupon Count Mean
High Yes 5 9.200
High No 5 7.400
Medium Yes 5 7.600
Medium No 5 4.800
Low Yes 5 5.400
Low No 5 2.000
TOTAL 30

Factor Level
Means
Promotion Coupon Count Mean
High 10 8.300
Medium 10 6.200
Low 10 3.700
Yes 15 7.400
No 15 4.733
Grand Mean 30 6.067
 One-Way ANOVA
This particular design is used when there is only one categorical independent
variable, and one dependent (metric) variable.
Each category of an independent variable is called a level. The independent
variable may be different levels of prices, or different pack sizes, or different
product colours, and the effect (dependent variable) could be sales, preferences or
attitudes towards the brand.
In the example that follows, we will look at advertising copy alternatives as the
independent variable, and preference rating for the advertising copy as the
dependent variable.
Worked Example Problem:
In this example, we assume that three different versions of advertising copy have
been created by an advertising agency for a campaign. Let us call these versions of
copy ADCOPY 1, 2 and 3. Now, the ad agency wants to test which of these three
versions of the advertising copy is preferred by its target population, before they
launch the campaign.

A sample of 18 respondents is selected from the target population in the nearby

areas of the city. At random, these 18 respondents are assigned to the 3 versions of
ad copy. Each version of ad copy is thus shown to six of the respondents.
The respondents are asked to rate their liking for the ad copy shown to them on a
scale of 1 to 10. (1 = Not liked at all, 10 = Liked a lot, and other values in between
these two). The ratings given by the 18 respondents are tabulated.
Slide 5

Input Data

Fig 1. shows the input data for the 18 respondents.

Fig. 1.

Sr. Ad rating
No. copy
1 1 6.00
2 1 7.00
3 1 5.00
4 1 8.00
5 1 8.00
6 1 8.00
7 2 4.00
8 2 4.00
9 2 5.00
10 2 7.00
 Slide 5 contd...

Fig. 1. Contd…

Sr. Ad rating
No. copy
11 2 7.00
12 2 6.00
13 3 5.00
14 3 5.00
15 3 4.00
16 3 7.00
17 3 8.00
18 3 7.00

The codes in the ‘ad copy’, column (1,2,3) indicate

the different versions of the ad. The last column,
‘rating’, is the rating given by a respondent to the
adcopy seen by him/her. Thus, six respondents have
rated each ad. Please note, that these eighteen
respondents were randomly assigned to each of the
three ad versions. This random assignment is called a
completely randomised assignment or design.
Slide 6

The input data in fig 1 is input into a statistical

package for performing a One-Way ANOVA,
because we have only 1 categorical factor (Ad copy)
at 3 levels – 1, 2, 3 and 1 dependent variable –
Rating.

Output

The output of the computerised One-Way ANOVA

is shown in fig. 2.
Fig. 2

Source of Sum of DF Mean F Sig.

Variation Squares Square of F

Main 7.000 2 3.500 1.780 .203

Effects
ADCOPY 7.000 2 3.500 1.780 .203
Explained 7.000 2 3.500 1.780 .203
Residual 29.500 15 1.967
Total 36.500 17 2.147
 Slide 6 contd….

The first column is titled ‘Source of Variation’. Under this, labeled Main
Effects, is the single independent variable called ADCOPY.

We then go to the last column, where the significance of the F test is given. It
is .203 in this case, for the factor ADCOPY. This indicates that at the
confidence level of 95 percent, (corresponding to significance level of 0.05),
the F-test proves the model is not significant. In other words, the Ratings
given to the three ad copy versions are not significantly different from each
other.
Slide 7

The ANOVA has thus told us what we may not have been able to gauge if we had
simply looked at the mean ratings for each ad copy by computing these.

For example, the ratings for the ad copy version 1 are 6,7,5,8,8,8 and the mean
rating is (6+7+5+8+8+8) / 6, or 42/6 = 7. Similarly, the mean rating of ad copy
version 2 is (4+4+5+7+7+6) / 6, or 33/6 = 5.5. The mean rating for ad copy
version 3 is (5+5+4+7+8+7) / 6, or 36/6 = 6.

At a glance, the three mean ratings appear to be different – 7, 5.5 and 6. But the
ANOVA tells us that this difference is not statistically significant at the 95 percent
confidence level.
It does this by performing an F-test. The null hypothesis for this F-test is that there
is no significant difference in the mean ratings for the three ad copy versions. (H 0:
M1 = M2 = M3 where M1, M2 and M3 are the mean ratings for the three versions of
ad copy). Thus, in this case, we have accepted the null hypothesis (or failed to
reject the null hypothesis), at the 95 percent confidence level.

If the significance of F in the last column of fig. 2 had been less than 0.05, we
would have rejected the null hypothesis. In that case, we would have concluded
that significant differences exist between mean ratings given to the three ad copy
 Slide 8
1. Randomised Block Design :
Let us continue with the same input data as in fig. 1,
with one more column added to it. This dataset is
shown in fig. 3.
Fig. 3
sr. adcopy rating magazine
no.
1 1 6.00 1
2 1 7.00 2
3 1 5.00 3
4 1 8.00 4
5 1 8.00 5
6 1 8.00 6
7 2 4.00 1
8 2 4.00 2
9 2 5.00 3
10 2 7.00 4
11 2 7.00 5
12 2 6.00 6
13 3 5.00 1
14 3 5.00 2
15 3 4.00 3
16 3 7.00 4
17 3 8.00 5
18 3 7.00 6
 Slide 8 contd…..
We have made a slightly different assumption in this
case. We assume that the three versions of the adcopy
were each used in 6 different magazines. These six
magazines are coded 1, 2, 3, 4, 5, 6 and appear in the
column titled “magazine”. Out of the people who saw
these ads, 18 randomly chosen respondents are
picked, one from each magazine who saw a particular
version of ad. Thus, we finally have one respondent
who has seen a given version of the ad in a given
magazine. In other words, we have one respondent
for every combination of magazine and adcopy.
Slide 9
Hypothesis

1. The assignment of our sample of 18 in the above manner assumes that the
magazine in which the version of adcopy appears may have an impact on the
ratings. We can test this hypothesis - in fact, two hypotheses - by doing an
ANOVA with a randomised block design.

2. For this purpose, we use the variable ‘Rating’ as the dependent variable, and
‘Adcopy’ as the factor, and ‘Magazine’ as the block.

3. A block is defined as some variable which could affect the relationship between
the independent factor and the dependent variable under study in an ANOVA. In
our example, the magazine in which the advertisement appears could influence the
Rating given to Adcopy by the respondents. We are trying to remove the effect of
the magazine used, by "blocking" its effect, or treating the block separately.

4. If we do not block on a variable, its effect gets included with the error (residual)
term. This may lead to wrong conclusions about the relationship between the
independent and dependent variables. In that sense, a randomised block design is
more "powerful" than a simple one-way ANOVA, if the block effect is
significantly influencing the relationship.
 Slide 10
Output
The computer output for this problem using a randomised block design is shown in
fig. 4.
Fig. 4
Tests of significance for RATING using UNIQUE sums of squares.
Source of SS DF MS F Sig
Variation of F
Residual 3.67 10 .37
Adcopy 7.00 2 3.50 9.55 .005
Magazine 25.83 5 5.17 14.09 .000
(Model) 32.83 7 4.69 12.79 .000
(Total) 36.50 17 2.15

This table is similar to the output table of the one-way ANOVA we got earlier (fig. 2),
except that there is an additional source of variation called ‘Magazine’ in the first
column of fig. 4. This is the ‘block’ we have used, to test the null hypotheses –

.The first null hypothesis is that mean rating of the ADCOPY is the same for all 3
versions. This is the same as the null hypothesis we had used earlier for the one-way
ANOVA.

.The second null hypothesis is that the ‘block’ used (Magazine in this case) has no
effect on mean ratings given to ADCOPY versions by respondents.
Slide 11

1. To test if the null hypotheses are rejected or not, we turn to the last column of fig.
4, which gives the result of an F-test for any assumed confidence level. We will
assume we wanted to test these hypotheses at the 95 percent confidence level.
2. We know that the significance level of F in the last column should be less than
0.05 for the null hypothesis to be rejected. We see that for both the rows labelled
ADCOPY and MAGAZINE, the significance of F is less than .05. It is .005 for
ADCOPY and .000 for MAGAZINE. This means that both the null hypotheses are
rejected.
3. We conclude that the mean ratings given to the 3 versions of ADCOPY are
significantly different, and also that the MAGAZINE in which the ADCOPY appears
has an impact on its rating.
4. Please note that the Blocking Factor being considered separately has now led us to
a different conclusion from that in a completely randomized test of the same basic
data. This makes the randomized block test a better test when we suspect that a
blocking factor affects the relationship between the independent variable and the
dependent variable.
 Slide 12

Latin Square Design

The Latin Square Design is an extension of the Randomised Block Design. It

consists of one independent variable (FACTOR) and two Blocks, instead of
one which we saw in the Randomised Block Design. It has no special
significance in marketing research, so we will move on to the more general
case of a factorial design where any number of factors can be tested
simultaneously for their effects on the dependent variable.

Factorial Designs

This type of design is employed when we have 2 or more independent

variables or factors. The major advantage of this design is that multiple
factors can be simultaneously tested. There are two kinds of effects that we
can test. One is called the Main Effect. The second is called the Interaction
Effect. To illustrate, we will take up an example.
Slide 13

Worked Example

In this example, we assume that we are testing for a toilet soap brand, the effect
of two Factors (independent variables) – Pack Design and Price - on Sales
(dependent variable). We would like to know (1) if each of the Factors
independently affects Sales (called the Main Effects), and (2) if there is a
combined effect of Pack Design and Price (called the 2 way Interaction Effect) on
Sales.

Incidentally, if there are 3 factors in a study, then we could test for all 2-way
interaction effects and the 3-way interaction effect, in addition to the Main
Effects of the individual factors.

To continue with our example, the experiment is conducted in a simulated

environment on 18 randomly selected respondents. There are 3 levels of price –
Rs. 8, Rs. 11 and Rs. 14, and 3 levels of Pack Design – designated by the main
colours used – Blue, Red and Green.

The coding of these variables is 1, 2, 3 respectively for Rs. 8, 11 and 14 and 1, 2,

3 for Blue, Red and Green in the case of Pack Design.
 Input Data

The input dataset is shown in .

sr. no. sales packdesn price
1 500 1 1
2 440 2 1
3 360 3 1
4 300 1 2
5 280 2 2
6 250 3 2
7 200 1 3
8 150 2 3
9 250 3 3
10 600 1 1
11 450 2 1
12 510 3 1
13 400 1 2
14 350 2 2
15 300 3 2
16 250 1 3
17 275 2 3
18 220 3 3
Column 1 is Sales, column 2 is Pack Design and Column 3 is Price. Please note that
even though Price is a continuous metric variable, for the purpose of ANOVA, being
an independent variable, it has to be treated as a categorical variable. Hence the
coding (1, 2, 3) for Price.
Slide 15
Also note from fig.5 that each combination of Price and Pack Design appears twice in
the dataset. For example, Packdesign = 1 and Price = 1 appears in Row 1 and also
Row 10. This is known as a replication in design of experiments. This is similar to
having a higher sample size in a survey.
Depending on the number of Factors and the number of levels of each Factor, the
minimum sample size required for ANOVA may go up. In such cases, multiple
observations or replications become necessary. In general, replications reduce
chances of random error affecting the results of ANOVA experiments, similar to the
effects of increasing sample size in surveys.
Output:
The output data
Source of for ourSum
Variation
factorial
of
Squares
experiment
DF areSquare
Mean presented inFfig. 6.SigF of
Fig 6Main 209305.556 4 52326.389 13.645 .001
Effects
12536.111 2 6268.056 1.635 .248
Packdesn
Price 196769.444 2 98384.722 25.656 .000
2-Way 9838.889 4 2459.722 .641 .646
Interactions
9838.889 4 2459.722 .641 .646
Packdesn
Price
Explained 219144.444 8 27393.056 7.143 .004
Residual 34512.500 9 3834.722
Total 253656.944 17 14920.997
Slide 16

Let us first look at Sources of Variation listed in the first column. The last
source of variation listed is the Residual or error term. But we are interested
in the two Main Effects and one Interaction Effect.

In this case, we are testing three hypotheses –

The mean level of Sales remains the same for all 3 levels of Pack
Design (Main Effect 1).
The mean level of Sales remains the same for all 3 levels of Price
(Main Effect 2).
The mean level of Sales remains the same for all combinations of
Pack Design and Price (Interaction Effect).

Assuming 0.05 level of significance, we check whether for each of the rows
corresponding to the above hypotheses, the significance of F is below 0.05
in the last column of fig. 6.
Slide 17

We find that the significance of F values are

Pack Design - .248 (Main Effect 1)

Price - .000 (Main Effect 2)
Pack Design by Price - .646 (Interaction Effect)

Therefore, only the Price effect, one of the two main effects, is significant
statistically, at 95 percent confidence level. This means that hypothesis no.
2 is rejected.

Hypothesis 1 and 3 cannot be rejected, as the significance of F values are

greater than .05 in both cases - .248 and .646 respectively).

Thus, we conclude that Price alone has an impact on Sales. Neither Pack
Design alone nor the combination of Pack Design with Price have any
significant impact on Sales of the toilet soap.
Slide 18 Additional Comments

Experiments are today widely used in many ways in Marketing Research. For
example, test marketing of new concepts, products or prototypes is usually done
through procedures explained above, or similar to these.
STM or simulated Test Marketing procedures are extensions of the basic ANOVA
type experiments, with the added tools of forecasting based on the results of
experiments conducts. Separate software packages are now available for many
specialised applications such as STM.

Pairwise Tests

If any main effect/interaction effect turns out significant, and has more than two
levels, there is one additional test required to check for pairwise differences in the
means.
For instance, in our example of one-way ANOVA, if the mean Ratings had turned out
to be significantly different at the 95 percent confidence level, we still would not
know whether only one of the pairs (say, ADCOPY 1 and ADCOPY 2) are
significantly different from each other, or if the remaining pairs (ADCOPY 1 and 3,
and ADCOPY 2 and 3) are also significantly different.
To find out, we can use tests such as Tukey's Test, Duncan's Test or Scheffe's Test.
These can be requested while doing the ANOVA on most computer packages. These
tests give us a pairwise test result of significant difference among means.
These are meaningful only if the F test value for a main effect/interaction effect with
more than two levels turns out to be significant.