Advanced and Multivariate

Statistical Methods
Practical Application and Interpretation
Sixth Edition

Craig A. Mertler and Rachel Vannatta Reinhart

© 2017 Taylor & Francis

 Chapter 5
Analysis of Covariance

© 2017 Taylor & Francis

 ANOVA vs. ANCOVA

• In social science research, it is difficult to imagine that differences on a

DV could only be attributed to the effect of one IV
• May be able to identify one or more variables that also demonstrate an
effect on DV
• However, if still only interested in the effect of the original IV, effects of
other concomitant variables can be controlled for or partialed out of
the results
o Variables whose effects have been partialed out of the results are called
covariates
© 2017 Taylor & Francis
 Analysis of Covariance
• More appropriate in social science research settings
• Extension of ANOVA where interaction and main effects are assessed
after effects of some other concomitant variable have been removed
o Effects removed by adjusting scores on DV to reflect initial differences on the
covariate
• Three purposes:
o Increasing the sensitivity of F tests of main effects and interactions by reducing
error variance (usually in experimental studies) – most common use
o Statistical adjustment procedure where means of DV are adjusted to what they
would be if all groups had scored equally on the covariate (usually in
nonexperimental studies where random assignment is not possible)
o Interpret differences in levels of IV when several DVs are included in analysis
© 2017 Taylor & Francis
 Analysis of Covariance
• Most common design involves a covariate consisting of scores on a
pretest measured in identical fashion as the DV (i.e., a posttest) but
prior to manipulation of levels of IVs
o Covariate does not need to be a pretest; can include demographic
characteristics, etc.
• For example, a two-way ANCOVA:
o Test three different null hypotheses: main effects for A, main effects for B,
interaction effects between levels of A and B, BUT only after effects of covariate
have been removed
1) Ho: µA1 = µA2 (H1: µA1 ≠ µA2) — main effects for A
2) Ho: µB1 = µB2 (H1: µB1 ≠ µB2) — main effects for B
3) Ho: µA1B1 = µA1B2 = µA2B1 = µA2B2 (H1: there is a significant interaction effect) —
interaction effect © 2017 Taylor & Francis
 Analysis of Covariance
• Choice of covariate is critical
o Ideally, any variables that theoretically correlate with DV or have been shown to
correlate with DV on similar types of participants should be considered possible
covariates
o Covariates should show strong correlations with the DV but have low
correlations among themselves (if more than one covariate is being used)
§ Weak correlation between two covariates will result in each removing a relatively
unique portions of the error variance from the DV
§ Strong correlations (r > .80) between covariates will remove essentially the same
error variability
Ø Therefore, no reason to include a second covariate

© 2017 Taylor & Francis

 Analysis of Covariance

• Number of covariates to include is also important

o Should be limited such that,

𝐶 + (𝐽 − 1)
< .10
𝑁

Where C is the number of covariates, J is the number of groups, and N is the

total number of participants

© 2017 Taylor & Francis

 Analysis of Covariance

• Sample research questions

o Are there significant mean differences for income (DV) between male and female
employees (levels of IV), after controlling for effects of the education level
(covariate)?
o Are there significant mean differences for income between individuals who are
satisfied and those who are dissatisfied, after controlling for effects of the
education level?
o Is there a significant interaction effect on income between gender and job
satisfaction, after controlling for effects of the education level?

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions (six) and tests [same three as ANOVA, plus three unique
to ANCOVA]
1) Observations within each sample must be randomly exampled and must be
independent of one another
§ A design issue; can't be statistically tested
§ Researcher must ensure random sampling and random assignment to believe that
this assumption has not been violated

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions and tests (cont’d.)

2) Distributions of scores on DV must be normal in the populations from which
they were sampled
§ ANOVA is generally robust to violations of normality assumption, especially if sample
size is large; not typically a concern
§ Even large deviations from normality will seldom have much of an effect on results
§ Distributions of the DV in each cell can be visually examined
§ Chi-square goodness-of-fit test can also be used

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions and tests (cont’d.)

3) Distribution of scores on the DV must have equal variances
§ Potential violation of this assumption is most crucial
§ Initially examined standard deviations in boxplots for each cell
§ If the visual evidence suggests a violation, several alternative tests may be
appropriate:
Ø Hartley’s Fmax test
Ø Cochran’s test
Ø Levene’s test
Ø Bartlett’s test

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions and tests (cont’d.)

4) Linear relationship exists between DV and covariate(s)
§ Only a concern when the covariate is quantitative
§ Roughly assessed by inspecting bivariate scatterplots between the covariate and DV
§ More precisely, examine residuals plots comparing standardized residuals to the
predicted values for the DV
§ Want to ensure that there is no curvilinear relationship present

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions and tests (cont’d.)

5) Regression slopes for a covariate are homogeneous (i.e., the slope for the
regression line is the same for each group) – homogeneity of regression
§ If the slopes are not equal, it is implied that there is a different DV-covariate slope in
some cells of the design or that there is an interaction between the IV and covariate
Ø If the latter occurs, relationship between covariate and DV is different at different levels of
IVs
Ø Covariate adjustment needed is different depending on the cell

© 2017 Taylor & Francis

Homogeneous regression slopes

© 2017 Taylor & Francis

Heterogeneous regression slopes

© 2017 Taylor & Francis

 Analysis of Covariance

• Assumptions and tests (cont’d.)

6) Covariate is reliable and measured without error
§ Essentially a design issue that should be addressed prior to data collection

© 2017 Taylor & Francis

 Analysis of Covariance

• Logic behind ANCOVA

o Nearly identical to the logic behind analysis of variance
o The only additional component is the adjustment of DV scores
§ Covariate is measured prior to manipulation of IV
§ Following implementation of treatments, dependent measures are collected
§ First part of analysis is to statistically adjust DV group means in order to control for
effects of covariates on DV
o Remainder of logic parallels that for either one-way or two-way ANOVAs

© 2017 Taylor & Francis

 Analysis of Covariance

• Interpretation of results
o Preliminary analysis includes testing for interaction between IVs and covariates;
homogeneity variance is also assessed
o If F test of factor-covariate interaction is significant, ANCOVA should not be
conducted
o Remainder of interpretation parallels that for ANOVA:
§ Examine line plots for graphical display of any factor interaction
§ Evaluate interaction significance using F ratio and p level from factorial ANOVA test
§ If no significant interaction is present, evaluate F ratio and levels of significance for
each factor’s main effect
§ Finally, examine and evaluate the effect size measures
© 2017 Taylor & Francis
 Analysis of Covariance

• Writing up results
o Begin with data screening and assumptions
o ANCOVA results – main effect of each factor and covariate, interaction of factors,
effect sizes
o Figures and tables (e.g., line graph of factor interaction, ANCOVA summary
table) may also be used to support narrative presentation

© 2017 Taylor & Francis

 Analysis of Covariance

• Sample study and analysis

o RQ1: Is there a difference in hours worked per week by gender among
employees when controlling for income differences?
o RQ2: Is there a difference in hours worked per week between satisfied and
unsatisfied employees when controlling for income differences?
o RQ3: Is there a difference in hours worked per week based on gender and level
of satisfaction when controlling for income differences?
• SPSS “how to”
o Follow screenshots and step-by-step directions in Chapter 5

© 2017 Taylor & Francis