Pek 2011

This article was downloaded by: [Central Michigan University]
On: 19 December 2014, At: 14:34

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,
UK
Multivariate Behavioral
Research
Publication details, including instructions for
authors and subscription information:
http://www.tandfonline.com/loi/hmbr20
Sensitivity Analysis in
Structural Equation Models:
Cases and Their Influence
a a
Jolynn Pek & Robert C. MacCallum
a
University of North Carolina at Chapel Hill
Published online: 19 Apr 2011.
To cite this article: Jolynn Pek & Robert C. MacCallum (2011) Sensitivity Analysis
in Structural Equation Models: Cases and Their Influence, Multivariate Behavioral
Research, 46:2, 202-228, DOI: 10.1080/00273171.2011.561068
To link to this article: http://dx.doi.org/10.1080/00273171.2011.561068
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the
information (the “Content”) contained in the publications on our platform.
However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness,
or suitability for any purpose of the Content. Any opinions and views
expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the
Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any
losses, actions, claims, proceedings, demands, costs, expenses, damages,
and other liabilities whatsoever or howsoever caused arising directly or
indirectly in connection with, in relation to or arising out of the use of the
Content.
This article may be used for research, teaching, and private study purposes.
Any substantial or systematic reproduction, redistribution, reselling, loan,
sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden. Terms & Conditions of access and use can be found at
http://www.tandfonline.com/page/terms-and-conditions
Downloaded by [Central Michigan University] at 14:34 19 December 2014
Multivariate Behavioral Research, 46:202–228, 2011
Copyright © Taylor & Francis Group, LLC
ISSN: 0027-3171 print/1532-7906 online
DOI: 10.1080/00273171.2011.561068
Sensitivity Analysis in
Structural Equation Models:
Cases and Their Influence
Jolynn Pek and Robert C. MacCallum

University of North Carolina at Chapel Hill
The detection of outliers and influential observations is routine practice in linear

regression. Despite ongoing extensions and development of case diagnostics in
structural equation models (SEM), their application has received limited attention
and understanding in practice. The use of case diagnostics informs analysts of
the uncertainty of model estimates under different subsets of the data and high-
lights unusual and important characteristics of certain cases. We present several
measures of case influence applicable in SEM and illustrate their implementa-
tion, presentation, and interpretation with two empirical examples: (a) a common
factor model on verbal and visual ability (Holzinger & Swineford, 1939) and
(b) a general structural equation model assessing the effect of industrialization on
democracy in a mediating model using country-level data (Bollen, 1989; Bollen &
Arminger, 1991). Throughout these examples, three issues are emphasized. First,
cases may impact different aspects of results as identified by different measures
of influence. Second, the important distinction between outliers and influential
cases is highlighted. Third, the concept of good and bad cases is introduced—
these are influential cases that improve/worsen overall model fit based on their
presence in the sample. We conclude with a discussion on the utility of detecting
influential cases in SEM and present recommendations for the use of measures of
case influence.
Mathematical models empower researchers with the ability to summarize, un-

derstand, explain, and predict complicated phenomena. Models are, however,
Correspondence concerning this article should be addressed to Jolynn Pek, Department of

Psychology, University of North Carolina, CB#3270 Davie Hall, Chapel Hill, NC 27599. E-mail:
pek@unc.edu
202
INFLUENTIAL CASES IN SEM 203
approximations of complex mechanisms and cannot be viewed as exact represen-

tations of real-world phenomena. Results from approximating models inherently
carry uncertainty in their results. For example, the same model fit to different
subsets of the data produces different results. The evaluation of uncertainty in
model results due to the introduction of small changes in modeling conditions
is termed sensitivity analysis (e.g., see Cook, 1986).
Useful results are grounded in models that achieve a close approximation
of the state of things, especially when results are stable, replicable, and inter-
pretable. Optimal results are derived from a set of modeling conditions that
capture major regularities in the data while remaining relatively insensitive to

noise. That is, reliable results should be robust or not highly sensitive to minor
changes or perturbations to modeling conditions.
Quantifying the impact of controlled perturbations to modeling conditions
on study results is an important diagnostic tool given its direct implications
on the inferences, interpretations, and conclusions drawn—especially if results
are unstable and sensitive to minor changes. With regard to the focus of this
article, it is highly desirable to ascertain that study results are not driven by
one or a few cases. Suppose that treatment efficacy of clinical therapy on
depression is confirmed from a set of modeling conditions; however, removing
a single case from the sample results in the opposite conclusion of no effect of
treatment. From this sensitivity analysis, study results are said to be sensitive
to the removal of an influential case, which reduces confidence in the original
findings. Information gained from sensitivity analysis following a systematic
case perturbation scheme thus (a) informs analysts of the uncertainty of model
estimates under different subsets of the data and (b) highlights characteristics
of certain important cases. The identification of an influential case indicates
that model estimates are sensitive to the presence of that particular case in the
sample and/or the case is highly inconsistent with the major regularities in the
data. Hence, the practice of identifying influential cases is not only beneficial
to evaluating model results but also to data screening and verification.
Case influence refers to the impact a case has on study results and is typically
quantified by deletion statistics. Deletion statistics compare solutions obtained
from the original sample with those obtained from the sample excluding case i ,
where i represents each case in turn. This approach to measuring case influence
assesses global influence of each case (Cadigan, 1995). Large deletion statistics
indicate that the net effect of deleting case i from the sample is substantial,
implying that the case in question exerts much influence on model estimates.
The identification of influential cases demands action that depends on the nature
of these cases and the intent of the analyst. An influential case arising from an
execution error requires data validation (Barnett & Lewis, 1994). In the context
of statistical analysis, valid influential cases may either be omitted by trunca-
tion or be incorporated via robust estimation techniques. Influential cases may
204 PEK AND MACCALLUM
also be subject to exploratory study. The observation of three white-eyed flies

among 1,237 red-eyed offspring led to the discovery of genetic heredity through
chromosomes (Morgan, 1910). More recently, the depletion of ozone levels over
Antarctica was discovered from observing case data that was characteristically
different from the rest of the sample (Farman, Gardiner, & Shanklin, 1985).
Detection of outliers and influential observations in the application of re-
gression analyses is fairly routine practice and often enhances the quality of
results of analyses. Yet, despite the extension and development of case influence
measures to structural equation models (SEM; e.g., see Lee & Wang, 1996;
Tanaka, Watadani, & Moon, 1991), the application of such diagnostics in SEM
has received limited attention in practice. The purpose of this article is to present
several measures of case influence adapted for SEM along with an approach for
presenting and evaluating findings obtained when such measures are computed
in empirical data. The measures we employ are adapted from regression analysis
and from literature on assessing case influence in the context of fitting falsifiable
models. We apply these methods in two empirical examples and we present
results that reveal several phenomena that may aid in model evaluation.
We highlight three key issues surrounding the diagnostic use and interpre-
tation of case influence measures in SEM. First, cases may exert influence
on different aspects of results, as revealed by the use of different types of
case influence measures. In particular, cases may independently exert influence
on model fit and/or parameter estimates, indicating the necessity of assessing
case influence using more than a single measure. Focus is thus placed on how
each measure of case influence uniquely adds to our understanding of case
characteristics and the impact these cases have on study results.
Second, as SEM generally involve overidentified models, cases may exert
influence on overall model fit. Distinctly different influential cases can substan-
tially improve or worsen model fit, allowing the investigator to identify and
examine what we refer to for simplicity as good and bad cases. Removing
good cases from the sample worsens model fit whereas removing bad cases
improves model fit. The identification of good and bad cases does not necessarily
mandate retention of the former and removal of the latter. In our empirical
examples, we recommend different approaches to addressing the presence of
these types of influential cases based on features of the case, the sample it
belongs to, and the models examined. In addition, we discuss the potential
value for using information about case influence on model fit as a means of
determining possible causes of poor model fit. We propose that such results can
potentially help to distinguish between instances where poor fit is caused by
poor model specification versus those where the cause may be associated with
the presence of a number of especially “bad” cases.
Third, the subtle but important distinction between outliers and influential
cases is highlighted. Although outliers often have considerable influence on
results of analyses, our results show that influence and outlying status are not
nearly perfectly associated. Outliers are not necessarily highly influential, and
influential cases are not necessarily outliers. As a result, we emphasize that
these issues should be treated separately in the application of SEM and that a
screening process involving the identification and elimination of outliers is not
sufficient to address the possible impact of highly influential cases.
Finally, we conclude with recommendations and a discussion regarding the
practical use of case influence measures in SEM. Our overall goal is that the
methods, findings, and recommendations we offer will assist users of SEM in
preventing the unhappy circumstance wherein important aspects of results and

interpretation may be driven by a very small number of influential cases, thus
compromising validity of findings.
CASE INFLUENCE IN STRUCTURAL

EQUATION MODELS
SEM is a general modeling framework where models of linear relations among

measured and latent variables are fit to a sample covariance matrix S of p
measured variables. Case influence in SEM may be quantified using different
types of deletion statistics, all of which require specifying a model. In this
article we present and demonstrate the implementation of two types of influence
measures adapted for SEM: (a) influence on overall model fit and (b) influence
on parameter estimates. These measures are model based as they quantify a
case’s effect on results obtained from fitting a model to the data. Without
loss of generality, we examine measures of case influence in SEM without
mean structure. The following developments remain applicable to models with
more complex structures. In addition, we examine the relationship between these
measures of case influence and a measure of outlier status.
Overall Model Fit

A general model-based measure of case influence on model fit is likelihood
distance (Cook, 1977, 1986; Cook & Weisberg, 1982) defined as
O
LDi D 2ŒL.™/ L.™O .i / /; (1)
where ™O and ™O .i / are the k 1 vectors of estimated model parameters based

on the original and delete i samples, respectively, where i D 1; : : : ; N . The
subscript .i / indicates that the estimate was computed on the sample excluding
case i . The log-likelihoods based on the original and delete i samples correspond
O and L.™O .i / /. Likelihood distance measures the magnitude and direction
to L.™/
of displacement in the likelihood function caused by removing case i from the

sample. Positive LDi values indicate poorer model fit associated with removing
case i as the log-likelihood of the full sample solution decreased. Conversely,
negative LDi values indicate that removing case i led to improved model fit
compared with the original sample.
The broad concept of likelihood distance may be adapted to SEM by im-
plementing Equation (1) and letting L represent some discrepancy function.
Under maximum likelihood (ML) estimation, a measure based on the concept of
likelihood distance LDi may then be developed from the following discrepancy
function:
FML D ln j†j ln jSj C trŒS† 1 p; (2)
where S and † are the p p sample and model implied population covariance
matrices, respectively. Rescaling the ML discrepancy function estimate FOML by
.N 1/ obtains the ¦2 test statistic under the Wishart distribution (Browne &
Arminger, 1995). The test statistic
¦2 D .N 1/FOML (3)
follows the ¦2 distribution with p.p C 1/=2 k degrees of freedom under the
null hypothesis that the model implied covariance matrix is no different from
the population covariance matrix. Note that k is the total number of parameters
estimated under the specified model. A test of this null hypothesis is often called
the test of perfect fit. Most measures evaluating overall model fit are based on
either the discrepancy function value in Equation (2) or the test statistic in
Equation (3). Holding the model constant, case influence on overall model fit
may be quantified by change in the test statistic
¦2i D ¦2 ¦2.i / ; (4)
where ¦2 and ¦2.i / are the test statistics obtained from the original and delete i
samples. Similar to the interpretation of LDi , but reversed in direction, positive
values on ¦2i suggest improved model fit and negative values on ¦2i indicate
poorer model fit when case i is removed. Indeed, it may be shown that LDi is
analogous to ¦2i as follows:
¦2i D .N 1/FOML .N O
2/FOML.i / D F .™/ .N O
2/ŒF .™/ F .™O .i //;
(5)
where F .:/ is the ML discrepancy function minimized to obtain estimates of ™O

or ™O .i / for the full and delete i samples.
Parameter Estimates
Cases may also be examined for their influence on parameter estimates where a
comparison is made between estimates obtained from the original and delete i
samples. Such a comparison assesses the difference between parameter estimates
based on the original and reduced samples. We describe here some measures of
case influence on parameter estimates parallel to the DFBETAS and Cook’s
Distance measures commonly used in regression analysis (Belsley, Kuh, &
Welsch, 1980; Cohen, Cohen, West, & Aiken, 2003; Cook & Weisberg, 1982).
Case influence on a vector of parameters may be quantified by the generalized

Cook’s Distance (gCD; Cook, 1977, 1986):
gCDi D .™O ™O .i //0 VAR.

O ™O .i / / 1 .™O ™O .i / /; (6)
where ™O and ™O .i / are l 1 vectors of parameter estimates obtained from the

original and delete i samples with l k. That is, if desired, we can calculate
gCD with reference to only a subset of k model parameters. Such an approach
may often be of interest in SEM when the investigator wishes to assess case
influence on a few .l / parameters rather than the full set of k parameters.
We recommend such an approach in practice. The l l estimated asymptotic
covariance matrix of the parameter estimates obtained from the reduced sample,
O ™O .i / /, serves to standardize the value of gCD. Due to its quadratic form,
VAR.
values on gCDi are bounded below by zero and will always be positive; smaller
values indicate that excluding case i led to smaller changes in estimates across
the l parameter estimates and larger values vice versa. Note that gCD conveys
information on the absolute magnitude of change in the vector of parameter
estimates but not the direction of change, caused by removing case i . Information
on the direction of change in parameter estimates may be obtained when case
influence on a single parameter is computed as follows:
™O j ™O j.i /
™O j i D ; (7)
O ™O j.i / / 12
ŒVAR.
where ™Oj and ™O j.i / denote the estimates of the j th parameter obtained from
the full sample and delete i samples. The denominator of Equation (7), which
standardizes the expression, is the standard error of the j th parameter obtained
from the sample excluding case i . Analogous to DFBETAS in regression, positive
values of ™O j i indicate that excluding case i caused a smaller estimate of ™O j
and negative values of ™O j i the converse. The expression in Equation (6) is a
squared scalar version of Equation (7):
gCDi D Œ™O j i 2 ; (8)
when case influence on a single parameter estimate j is assessed using gCD.

Outlying Status
Cases may also be examined for outlying status, which is distinct from influential
status. A case is termed an outlier when it is observed to be aberrant compared
with other observations to the point of raising concerns that it belongs to a
different population (Hawkins, 1980; Rasmussen, 1988). Outlier detection is an
important task in data analysis in SEM as extant work confirms that outliers
can potentially create Heywood cases or improper solutions (Bollen, 1987). A
small proportion of outliers in a sample can substantially distort results under
ML and asymptotically distribution free estimation (Browne, 1984) leading to

biased parameter estimates and spuriously significant test statistics (Yuan &
Bentler, 2001).
The most common measure for identifying multivariate outliers is the Maha-
lanobis Distance (MD; Mahalanobis, 1936) defined as
MDi D .xi x/0 S 1 .xi x/; (9)
where xi is the p 1 vector of p observed variables for case i , x is the p 1

vector of sample means, and S 1 is the inverse of the p p sample covariance
matrix. MD is interpreted as the squared distance between case i ’s data and the
centroid of all cases adjusted with respect to the variances and covariances of
the observed variables.
Under multivariate normality, MDi is distributed as ¦2 with p degrees of
freedom. Some researchers advocate formally testing cases for outlying status
using MD (e.g., Rousseeuw & Van Zomeren, 1990; Tabachnick & Fidell, 2001).
By choosing some nominal error rate ’, cases with MDi values exceeding the
¦21 ’ .p/ quantile would be considered multivariate outliers under such a decision
rule. Inherent in this test is the assumption that the p variables follow a mul-
tivariate normal distribution. Additionally, values larger than this critical value
may not necessarily be outliers but legitimately belong to the data distribution.
Outlying status of a case may also be understood as a case’s influence on
the sample covariance matrix S. Belsley et al. (1980) introduced the covariance
ratio (COVRATIO) as a means of quantifying the effect of deleting a case on
the covariance matrix:
COVRATIO D jS.i /j=jSj; (10)
where jS.i /j and jSj are the determinants of the p p sample covariance matrix
after omitting case i and using the complete sample, respectively. COVRATIO
values close to unity may be taken to indicate that the two covariance matrices
are not substantially different.
A comparable measure of a case’s impact on the sample covariance ma-
trix may be developed using the ML discrepancy function in Equation (2).
In its conventional usage this function measures the discrepancy between the
sample covariance matrix and the implied population covariance matrix. We
have explored the possibility of using this function to measure instead the
difference between the sample covariance matrix S based on the full sample
and a covariance matrix S.i / based on the deletion of case i from the sample:
Si D ln jS.i /j ln jSj C t r .S.i / S 1 / p; (11)
where Si is the change in the sample covariance matrix S brought about by
the removal of case i . Cases with Si values close to zero indicate that their
removal from the sample does not bring about large changes in the sample
covariance matrix.
Analytical relationships among MD, COVRATIO, and Si , along with results
from empirical applications, confirm that all three measures of outlying status
are almost perfectly correlated. In particular, MD is virtually perfectly negatively
correlated with COVRATIO and almost perfectly positively correlated with Si .
As COVRATIO and Si do not significantly add to our study of case influence
and outlying status, they are not examined further in this article.
Influential Cases Versus Outliers

Contrary to popular assumption, outlying cases do not necessarily exert strong
influence on results of statistical analyses. Outliers are identified by their deviant
status with respect to sample characteristics. No model is explicitly fit to the data
in the identification of outliers. Influential cases are, however, defined by their
impact on results of a model fit to the data. Influential cases may impact results
either at the level of overall model fit or at the level of parameter estimates.
As demonstrated in the next section, these properties of cases may function
independently in that an influential case may not necessarily be an outlier and
vice versa. Although some correspondence between outliers and influential cases
is to be expected, a one-to-one mapping cannot be assumed. Therefore, as we
show in our examples, the practice of using a measure such as MD to screen
for and then delete outliers so as to avoid their influence on findings may not
be highly effective in practice in SEM and may leave some highly influential
cases in the sample.
TWO EMPIRICAL APPLICATIONS
To illustrate and highlight several issues involved in interpreting measures of case

influence in SEM, sensitivity analysis following a case deletion perturbation
scheme was conducted on two real-data examples. The first example fits the
common factor model to six psychological tests (Holzinger & Swineford, 1939)
and the second example fits a mediation model among three latent variables on
country-level data (Bollen, 1989). All measures of case influence and outlying
status were computed exactly. Exact computations of measures of influence
involve the systematic task of obtaining N sets of delete i sample results and
comparing them with results based on the original sample.1 Outlying status
is computed exactly from Equation (9), which is expressed in closed form.
The following sections provide more detail on the examples and illustrate the
diagnostic use of the different case influence measures.
Six Psychological Tests

A common factor model was hypothesized to explain correlations between
six psychological tests obtained from a sample of 301 seventh- and eighth-
grade children (Holzinger & Swineford, 1939). A visual perception factor was
expected to explain scores on visual perception .y1/, cubes .y2/, and lozenges
.y3/, and a verbal ability factor was expected to explain scores on paragraph
comprehension .y4/, sentence completion .y5/, and word meaning .y6/. The
visual perception test is a non-language test where items were selected from
Spearman’s Visual Perception Test, Part III. The cubes test is a simplified version
of one of Brigham’s tests of spatial relations. The lozenges test was adapted
from a test prepared by Thurstone (1938, p. 32), which was hypothesized to
measure kinesthetic imagery. The paragraph comprehension and word meaning
tests were Part III and Part II of the Traxler Silent Reading Test, respectively
(Traxler, 1934). Finally, the sentence completion test was a test adapted from
the Unitary Trait Study. The common factor model in covariance structure form
is expressed as
† D ƒ‰ƒ0 C ‚; (12)
where † is the p p population covariance matrix, ‰ is the m m correlation

matrix between the factors, and ‚ is the p p matrix of unique variances.
The p m factor loading matrix ƒ contains the p slopes (loadings) of the
regression for the observed variables onto the m common factors. The matrix
1 For these illustrations, computation of influence measures was carried out using a combination
of R code, Mplus batch runs, and SAS code. Therefore, it is not feasible at this point to provide
generally useful code to interested researchers. As we develop and refine software to implement
the methods illustrated in this article, we will provide online and e-mail support to interested users.
Jolynn Pek may be contacted and we will provide our current software implementation for use in
applied research. In the future, we may be able to provide user-friendly code, or SEM software
developers might consider providing case influence measures in commercial software if there is
sufficient interest.
‚ is constrained to be diagonal, reflecting the assumption that the observed

variables are independent, conditioned on the common factors.
Two models with increasing constraints were fit to the data. Model A is a
restricted two-factor model with correlated factors where
2 3
œ1;1 0
6œ2;1 0 7
6 7
6œ3;1 0 7
ƒA D 6 6 0
7
œ4;2 7
6 7
4 0 œ5;2 5
0 œ6;2

1:0
‰A D and ‚ A D diagŒ™1 ; ™2 ; ™3 ; ™4 ; ™5 ; ™6 :
§2;1 1:0
The first and second factors correspond to the visual perception and verbal
factors, respectively. Model B is a restricted two-factor model with independent
factors where
2 3
œ1;1 0
6œ2;1 0 7
6 7
6œ3;1 0 7
ƒB D 6 6 7
6 0 œ4;2 7
7
4 0 œ5;2 5
0 œ6;2

1:0 0
‰B D and ‚ B D diagŒ™1 ; ™2 ; ™3 ; ™4 ; ™5 ; ™6 :
0 1:0
Constraining §2;1 D 0 in Model A obtains Model B, implying that the models are
nested and that Model A necessarily fits the data better than Model B. To amplify
the effect of outliers and influential cases for purposes of this demonstration
and to simplify the results, the original sample of No D 301 was reduced to
N D 100 by retaining the 10 cases with the largest MD values and randomly
drawing 90 cases falling below the 75th percentile of MD. Standard scores on
the six tests and MD for these 10 cases (numbered 91 to 100) from the reduced
sample are presented in Table 1. Based on a nominal error rate of ’ D 0:05, a
¦2 .6/ distribution is associated with a critical value of 12.592 and 9 of the 10
cases in Table 1 would be flagged as potential outliers because their MD values
exceed this threshold. Table 2 presents model estimates and model fit information
obtained from the unperturbed sample .N D 100/. According to the test of
perfect fit, Comparative Fit Index (CFI; Bentler, 1990), Tucker-Lewis Index
(TLI; Tucker & Lewis, 1973), and Root Mean Square Error of Approximation
TABLE 1
Z Scores and Mahalanobis Distances (MD) for the 10 Cases With Largest MD
Case y1 y2 y3 y4 y5 y6 MDi
91 1.432 1.420 1.719 0.817 1.039 0.904 16.268

92 2.016 2.077 1.593 2.288 3.251 1.155 21.774
93 1.717 2.515 1.936 0.523 1.039 2.356 20.668
94 2.166 0.543 1.593 2.417 1.716 3.359 13.434
95 0.967 1.858 2.050 2.123 1.490 3.108 12.144
96 1.732 1.200 0.120 2.711 1.264 2.105 17.932

97 1.282 3.400 0.463 1.535 1.490 1.729 21.010
98 3.216 0.324 1.822 2.417 1.716 2.732 14.774
99 1.282 0.324 0.120 0.653 0.542 2.105 16.418
100 2.031 2.077 1.593 1.241 0.813 0.977 18.249
(RMSEA; Browne & Cudeck, 1993; Steiger & Lind, 1980), Model A fits the
data well but not Model B.
Measures of influence and outlying status are typically depicted using index
or casewise plots where case numbers are presented on the abscissa and values of
the measure of influence or outlying status are presented on the ordinate. Within
each index plot, cases with extreme values are typically visually identified as
influential cases or outliers by their departure from the general level and range
of points. Figure 1 presents an index plot of Mahalanobis Distance (MD) that
conveys information on cases’ outlying status. The lower horizontal reference
line marks the expected value of a ¦2 .6/ distribution and the higher horizontal
reference line marks the critical value of ¦20:95 .6/ D 12:592; all nine points
lying above the latter horizontal line may be considered multivariate outliers.
Additionally, the vertical reference line distinguishes the potential outliers, to
the right, from the remaining sample. Observing that a majority of MD values
fall below the expected value is consistent with the positive skew of the ¦2
distribution. From Table 1 and Figure 1, Case 92 has the largest MD followed
by Cases 97 and 93, suggesting that these cases are potential outliers. Stated
differently, removing these cases substantially changes the sample covariance
matrix S.
Figure 2 presents index plots of case influence on overall model fit .¦2 / for
Models A and B. The removal of Case 97 is associated with the largest absolute
change in the ¦2 test statistic. The influence of Case 97 on overall model fit
is consistent across the two models in that ¦2 is negative (¦2A D 3:448
and ¦2B D 4:363) meaning that removing Case 97 worsens the fit of Models
A and B. However, case influence on overall model fit may not operate in the
same direction across models, as illustrated by Case 92. Removing Case 92
improved overall fit in Model A .¦2A D 1:326/ but worsened overall fit in
TABLE 2
Parameter Estimates and Model Fit Information
for Models A and B (N D 100)
Model A Model B
Parameter Estimate Estimate
œ1;1 0.587 0.539

œ2;1 0.523 0.489
œ3;1 0.890 0.969

œ4;1 0 0
œ5;1 0 0
œ6;1 0 0
œ1;2 0 0
œ2;2 0 0
œ3;2 0 0
œ4;2 0.881 0.890
œ5;2 0.828 0.834
œ6;2 0.817 0.801
§2;1 0.352 0
™1 0.656 0.709
™2 0.726 0.760
™3 0.208 0.061
™4 0.224 0.209
™5 0.315 0.305
™6 0.332 0.358
¦2 (df) 11.902 (8) 20.671 (9)
CFI 0.984 0.952
TLI 0.970 0.919
RMSEA (90% CI) 0.070 (0, 0.148) 0.105 (0.034, 0.172)
Note. Model A D restricted factor analysis with correlated factors.

Model B D restricted factor analysis with independent factors. Values of
zero correspond to fixed parameters. CI D Confidence Interval; CFI D
Comparative Fit Index; TLI D Tucker-Lewis Index; RMSEA D Root Mean
Square Error of Approximation.
Model B .¦2B D 4:003/. The extent and direction of influence of a case on

overall model fit communicates the consistency the case in question has with the
model. Removing Case 97 from Models A and B resulted in worsened model
fit, suggesting that Case 97 is consistent with the models. Case 92, however,
is somewhat inconsistent with Model A but consistent with Model B. With
respect to case influence on overall model fit, the example of Case 97 confirms
the supposition that outliers have considerable influence on study results. Yet,
the influence of the potentially outlying Case 92 on the fit of Model A was
comparable to that of many of the non-outlying cases (see Figure 2).
FIGURE 1 Index plot for Mahalanobis Distance. Note. Cases above the higher horizontal
reference line may be regarded as potential multivariate outliers.
The direction of influence a case exerts on overall model fit distinguishes

good cases from bad cases. Recall that positive/negative values on ¦2i imply
better/poorer model fit when case i is removed. In this vein, cases with negative
values on ¦2i are good in that their removal worsens model fit and conversely,
cases with positive values on ¦2i are bad. In Figure 2, an almost equal number
of good and bad cases are observed where good/bad cases are identified be-
low/above the horizontal reference line. It is important to note that case definition
of good/bad is model dependent. Case 97 is a consistently good case whereas
Case 93 is a consistently bad case for Models A and B. Interestingly, Case 92
was a moderately bad case for Model A .¦2A D 1:326/ and an extremely
good case for Model B .¦2B D 4:003/. This observation prompts a close
inspection of the data for this case. Data shown in Table 1 for Case 92 showed
large positive scores on the visual tests and large negative scores on the verbal
tests, supporting a strong negative correlation between the visual perception and
verbal factors. But results for Model A show that the estimate of the correlation
between the two factors is positive .r D :35/. Clearly, the regularity in the data
represented by the bulk of the sample supports a positive correlation between the
factors, whereas the data for Case 92 contradicts that regularity, making Case
92 a “bad” case for Model A. For Model B, which posits uncorrelated factors,
Case 92 is a “good” case because, as seen later, its presence pushes down the
estimate of the factor intercorrelation and thus causes Model B to fit better. A
FIGURE 2 Case influence on overall model fit .¦2 /. Note. k D number of parameters
estimated. Cases with large Mahalanobis Distances fall to the right of the vertical dotted
reference line. Cases above and below the horizontal reference line are bad and good cases,
respectively.
prudent next step would be to verify the integrity of these influential cases’ data
given their impact on overall model fit. If there were no errors in the data, a
decision may be made to include/exclude these cases from the sample when
estimating models of interest. There may also be substantive value in finding an
explanation behind Case 92’s interesting pattern of test scores.
We now turn to results for measures of influence on parameter estimates.
Index plots for generalized Cook’s Distance (gCD), depicting case influence on
sets of parameter estimates, for Models A and B are presented in Figure 3. In this
figure gray open circles represent influence on the full set of parameter estimates,
and black dots represent influence on a subset of parameters consisting of only
FIGURE 3 Case influence on a vector of parameter estimates (Generalized Cook’s

Distance). Note. k D number of parameters estimated; l D number of factor loadings.
Cases with large Mahalanobis Distances fall to the right of the vertical dotted reference line.
factor loadings. The potential outliers, located to the right of the reference line,
tend to exhibit more influence on the examined sets of parameter estimates
regardless of model. In Model A, Case 92 had the largest influence on both sets
of parameter estimates, suggesting that Case 92 may be inconsistent with the
structure imposed by Model A. In Model B, where the correlation between the
factors was constrained to zero, Case 93 exerted the largest influence on the
full set of parameter estimates whereas Case 92 had the largest influence on the
factor loading estimates, implying that Case 92 had little impact on the residual
variances. Case 97 had a large impact on the k estimates in Model B but a
relatively small impact on the l factor loadings, suggesting that the influence
Case 97 has on parameter estimates in Model B concerns the residual variances.
Beyond case influence on overall model fit, gCD provides summary information
that communicates cases’ consistency with the hypothesized model in terms of
their influence on some specific set of parameter estimates.
Case influence on a set of parameter estimates may be examined across
models with the recommendation that the number of parameters used to compute
gCD is held constant. The mean and variance of gCD are positively associ-
ated with the number of parameter estimates assessed as gCD asymptotically
follows an F .k; N k/ distribution. If we compared gCD on the complete
set of parameter estimates between Models A and B (depicted by the gray
open circles in Figure 3) without regard to the number of parameters, we

might erroneously conclude that cases in Model A are more influential than
Model B. Indeed, when gCD is computed using the same number of parameters
across models, the influential impact of cases will be interpreted on the same
scale. As shown by the black dots in Figure 3 for the l factor loadings, case
influence values between the models are more comparable as they are in the
same scale.
The use of gCD provides an efficient approach to examine case influence on
subsets of estimates, especially in the context of complex models with many
parameters. However, gCD lacks specificity in identifying parameter estimates
within the defined set which are most sensitive to influential cases. Additionally,
gCD provides no information on the directionality of influence caused by the
case. Instead, case influence on a specific parameter estimate as defined in
Equation (7) may be computed to address these limitations. Figure 4 presents the
index plot of case influence on the factor correlation estimate .§ O 2;1 / of Model
A. The removal of Case 92 led to the largest standardized change in § O 2;1 where
§O 2;1 D 1:047, implying that the absence of Case 92 caused about a one
standard unit increase in the estimate of §2;1 .
Cases may exert influence on different aspects of results such as on overall
model fit .¦2 /, on a set of parameters (gCD), or on a specific parameter
.™O j /. The correspondence between these different measures of influence is
complex as illustrated by Case 92. In Model A, Case 92 did not exert much
influence on overall model fit as shown in Figure 2 but had the largest impact
on all parameter estimates measured by gCD in Figure 3 and the largest effect
on § O 2;1 depicted in Figure 4. We noted earlier that scores on the measured
variables for Case 92, presented in Table 1, indicated high scores on the visual
tests and low scores on the verbal tests, supporting a strong negative correlation
between factors. Case 92 thus had a strong influence on § O 2;1 , opposed to the
rest of the sample, and removing it understandably led to a large change in
the parameter estimate § O 2;1 . Hence, the combination of information from the
different measures of case influence suggest that Case 92 is inconsistent with
the sample, and the general structure of Model A, especially at the level of the
factor correlation estimate §O 2;1 .
FIGURE 4 Case influence on the factor correlation estimate § O 2;1 in Model A. Note. Cases
with large Mahalanobis Distances fall to the right of the vertical dotted reference line.
Removing cases above/below the horizontal reference line led to smaller/larger estimates of
§2;1 .
Across Figures 1 through 4, potential outliers (located to the right of all

vertical reference lines) tended to exhibit the most influence on overall model
fit and parameter estimates. However, outlying cases may not necessarily be
influential as illustrated by Case 100. From Table 1, Case 100 had the fourth
largest MD value but had little impact on overall fit for Models A and B
(Figure 2) and the parameter estimate §O 2;1 (Figure 4). The close correspondence
between outlying status (quantified by MD in Figure 2) and case influence on
some set of parameter estimates (measured by gCD in Figure 3) seems to suggest
that outlying cases are influential. However, much of the association between
MD and gCD stems from the similar quadratic forms of their defining Equations
(6) and (9), with the former summarizing case characteristics at the level of the
measured variables and the latter at the level of the parameter estimates.
Industrialization and Democracy

The second application demonstrates that these measures of case influence and
outlying status may be applied to the general SEM in an unmodified data set.
Data were available for N D 75 countries that provided information on three
latent variables: political democracy in 1960 and 1965 as well as industri-
alization in 1960. Industrialization in 1960 was indicated by three measured

variables: gross national product per capita .y1/, energy consumption per capita
.y2/, and percentage of the labor force in industrial occupations .y3/. Political
democracy for 1960 and 1965 was measured using the same four indicators:
freedom of the press .y4; y8/, freedom of group opposition .y5; y9/, fairness
of elections .y6; y10/, and elective nature and effectiveness of the legislative
body .y7; y11/.
Industrialization is known to promote the chances for a democratic political
system. In the two models (C and D) fit to the data, industrialization in 1960 .˜1 /
is specified to contemporaneously influence democracy in 1960 .˜2 / and in 1965

.˜3 /. Additionally, the measurement errors of the same democracy indicators
were allowed to correlate over time (™4;8 ; ™5;9 ; ™6;10 and ™7;11 ). To identify the
models, all latent variables were scaled according to their first indicators by
fixing respective factor loadings to 1.0. Assuming measurement invariance for
political democracy across the two time points, the factor loadings for the three
remaining indicators of political democracy were constrained to be equal across
each measurement period (œ5;2 D œ9;3 D œ5 , œ6;2 D œ10;3 D œ6 and œ7;2 D
œ11;3 D œ7 ). To account for a common data source, the measurement errors of
freedom of group opposition .y5; y7/ and elective nature and effectiveness of
legislative body .y9; y11/ were correlated (™5;7 and ™9;11 ). Figure 5 presents a
path diagram of Model C fit to the data. By constraining ™5;7 D ™9;11 D 0 in
Model C, we obtain Model D, which does not account for the common data
source. Model D is nested in Model C, such that Model C will necessarily fit the
data better than Model D. A more detailed description of the data and the model
are available in Bollen (1989) and Bollen and Arminger (1991). Estimates for
Model C based on N D 75 are presented in Bollen (1989, p. 335).
Overall, both models fit the original sample fairly well, with Model C fitting
significantly better than Model D, ¦2 .2/ D 13:704, p < :01. Fit information
for Model C is ¦2 .38/ D 40:170, ns, CFI D 0.997, TLI D 0.995, and RMSEA D
0.028 (90% confidence interval [CI] D 0; 0.087). For model D, ¦2 .40/ D 53:874,
ns, CFI D 0.979, TLI D 0.972, and RMSEA D 0.068 (90% CI D 0; 0.111).
Figure 6 presents an index plot of MD with the horizontal reference line marking
the critical value of a ¦2 .11/ distribution with an error rate of ’ D 0:05. Cases
45 and 68 exceed the critical value of 19.675 and are flagged as potential outliers
with MD45 D 21:229 and MD68 D 21:766.
Figure 7 presents index plots of case influence on overall model fit .¦2 /
for Models C and D in the top and bottom panels, respectively. There is greater
dispersion of ¦2 in the poorer fitting Model D compared with the better fitting
Model C as models that fit extremely well will have few or no influential cases
on overall model fit whereas poorer fitting models will tend to have more cases
with large values of ¦2 . Because Model D fits more poorly than Model C, it
necessarily has more influential cases on overall model fit.
FIGURE 5 Industrialization .˜1 / and democracy (˜2 and ˜3 ) mediation model for
developing countries, 1960 to 1965 .N D 75/.
Further study of the results in Figure 7 reveals an interesting phenomenon

that provides for a criterion for evaluating the degree of influence on model
fit. Specifically, influential cases may potentially affect the outcome of a test
of model fit. The test of perfect fit for Model D yielded ¦2 D 53:874 with 40
degrees of freedom; setting ’ D 0:05, the critical value for this test statistic
FIGURE 6 Index plot of Mahalanobis Distance. Note. Cases above the horizontal reference
line may be regarded as potential multivariate outliers.
is 55.758, and the hypothesis of perfect fit is not rejected. Note, however, that
removing any case i with ¦2i 1:884 will lead to a significant test statistic
and rejection of the hypothesis of perfect fit. The nonzero horizontal reference
line in Figure 7 for Model D depicts this cutoff and identifies nine such cases.
For example, removing Case 45 results in ¦2.45/ D 58:549, p D :02929. Clearly,
the outcome of this test of fit of Model D is strongly affected by the presence
of this handful of “good” cases. Of course, this phenomenon could operate in
reverse, where the deletion of a single “bad” case could change the outcome of
such a test from rejection to non-rejection of perfect fit. Although not illustrated
in this article, cases may also potentially affect the outcomes of other tests of
model fit (e.g., the commonly used test of close fit) as well as tests of the
significance of key parameter estimates.
The potential outliers of Cases 45 and 68 (labeled by their case numbers)
in Figure 7 are good for Models C and D in that their removal is associated
with negative ¦2 , which translates to poorer model fit. Hence, removing these
potential outliers before fitting models as suggested by some (e.g., Tabachnick &
Fidell, 2001) may be detrimental to the modeling process. Interestingly, the most
influential case for Model C is not an outlier. Case 74 exerted the largest absolute
influence on overall fit for Model C .¦274 D 4:282/ but had a relatively
small MD estimate of MD74 D 10:868. Similarly, the most influential case for
FIGURE 7 Case influence on overall model fit .¦2 /. Note. k D number of parameters es-
timated. Numbered cases have significantly large Mahalanobis Distances. Cases above/below
the zero horizontal reference line are bad/good cases. Removing cases below the nonzero
horizontal reference line leads to rejection of the test of perfect fit.
Model D is not an outlier; Case 14 had ¦214 D 4:741 and MD14 D 15:783.
Therefore, influential cases are not necessarily outliers, suggesting that influence
and outlying status may not be nearly perfectly associated. However, outliers tend
to be influential and Cases 45 and 68, which are potential outliers based on MD,
exerted substantial influence on overall model fit for both models as shown in
Figure 7. Although most cases influenced overall model fit in the same direction
for the two models, Case 14 was a good case for Model D .¦214 D 4:741/
but had little to no impact for Model C .¦214 D 0:384/; after accounting for
the common data source for freedom of group opposition .y5; y9/ and elective
nature and effectiveness of legislative body .y7; y11/, Case 14 was no longer
influential on overall model fit.
The study of case influence on parameter estimates not only identifies un-
usual cases but also communicates information on the stability of theoretically
important estimates. Because promoting the chance for a democratic political
system by industrialization is a key finding, robustness of the contemporaneous
effects of industrialization on democracy in 1960 .“O 2;1 / and 1965 .“O 3;1 / to case
influence are examined. As Model C significantly fits the data better than Model
D, only case influence on the direct effect .“O 3;1 / and indirect or mediation effect
.“O 2;1 “O 3;2 / in Model C are highlighted. The index plot in Figure 8 shows that
case influence on the direct effect of industrialization in 1960 on democracy in
FIGURE 8 Case influence on the direct effect “O 3;1 in Model C. Note. Numbered cases
have significantly large Mahalanobis Distances.
1965 .“O 3;1 / for Model C ranged between 0.517 and 0.500. Under the delete-one
perturbation scheme, this direct effect estimate did not fluctuate beyond ˙ 0.600
standard units. Although Cases 45 and 68 were identified as potential outliers,
both had little influence on “O 3;1 . Since the standardized estimate based on the
original sample of “O 3;1 D 2:520, this important effect remained statistically
significant at ’ D 0:05. The study of case influence on “O 3;1 therefore confirms
that this key estimate is relatively robust to casewise deletion.
Case influence on the indirect effect of industrialization on democracy in
1965, mediated by democracy in 1960, was assessed using the Sobel test statistic
(Sobel, 1982; Baron & Kenny, 1986) defined as
“O 2;1 “O 3;2
zD ; (13)
Œ.“O 3;2 /2 VAR. O “O 3;2 / 21
O “O 2;1 / C .“O 2;1 /2 VAR.
where “O 2;1 is the effect of industrialization on democracy in 1960 and “O 3;2

is the effect of democracy in 1960 on democracy in 1965. The denominator
standardizes the expression where VAR. O “O 2;1 / and VAR.
O “O 3;2 / are the asymptotic
variances associated with their respective direct effects. The Sobel test statistic
asymptotically follows a normal distribution and formally tests whether the
mediator (democracy in 1960) carries the influence of industrialization in 1960
to democracy in 1965. To assess case influence on this mediation effect, a
deletion statistic based on the Sobel test was computed for each delete i sample
as follows:
“O 2;1 “O 3;2 “O 2;1.i / “O 3;2.i /

zi D ; (14)
Œ.“O 3;2.i //2 VAR. O “O 3;2.i // 21
O “O 2;1.i / / C .“O 2;1.i / /2 VAR.
where the subscript .i / indicates that case i was excluded from the sample.
Note that this measure of case influence on the Sobel test is analogous to case
influence on a single parameter as defined in Equation (7). Positive/negative
values of zi indicate the direction and magnitude (in standard units) of change
in the Sobel test statistic caused by excluding case i . The significant mediation
effect obtained from the original sample, z D 3:570, p < :0001, was found to
be robust under the delete i perturbation scheme as 0:379 < zi < 0:265.
Although Cases 45 and 68 were identified as potential multivariate outliers,
they had little influence on the estimate of the indirect effect, indicating that
outliers may not be influential on certain aspects of model results. Given that
key parameter estimates for Model C are robust to casewise deletion and overall
model fit is good, Model C presents itself as a strong plausible model that closely
approximates the phenomena under study.
DISCUSSION AND RECOMMENDATIONS
In this article we have presented and applied a number of measures of case influ-
ence adapted for SEM. Our results show the potential utility of such sensitivity
analysis of cases for assessing the quality of study results and for highlighting
and examining characteristics of key cases in the sample. At a basic level, this
process aids in establishing data integrity. Unusual cases due to coding or other
errors of execution (Barnett & Lewis, 1994) can be identified and corrected or
deleted, whereas valid but atypical cases may warrant further exploration.
More important for present purposes, the careful assessment of case influence
may serve the critical purpose of helping the investigator to avoid drawing and
reporting conclusions that are strongly driven by one or a few highly influential
cases. The measures we have presented and the results from our demonstrations
support this perspective and provide validation of several points emphasized
early in this article. These include, first, the need to use multiple measures
of case influence because cases may influence difference aspects of results,
especially model fit and estimates of key parameters. Cases that exert little or
no influence on one aspect may show a strong influence on another aspect.
Second is the need to closely examine “good” and “bad” cases with regard to
influence on model fit. We demonstrated how a single case can alter the outcome
of a test of fit. We also demonstrated how one can investigate the nature of a
particular case and reach an understanding of how that case is inconsistent
with a specified model. We also wish to emphasize that the pattern of findings
regarding good and bad cases might provide insight into the causes of a finding
of mediocre to poor model fit. When a model is generally poorly specified and
inconsistent with the population of interest, we expect that an analysis of case
influence would show many or most cases contributing to poor fit. But when case
influence measures show only a few extremely bad cases, then an investigator
may be justified in believing that the model itself is perhaps not poorly specified
and that the problem lies in the fact that the sample contains a few highly atypical
cases that are inconsistent with the major regularities in the data.
The third point of emphasis involves the distinction between influential cases
and outliers. Although outliers are often influential, they are not always so, and
highly influential cases may not be outliers. Results in our examples illustrate
this phenomenon, which clearly arises because measures of influence are model
based and measures of outlying status are not. Thus investigators must recognize
the desirability of examining influence in addition to outlying status of cases.
Furthermore, the practice of truncating or censoring potential outliers prior
to model fitting (Rousseeuw & Van Zomeren, 1990; Tabachnick & Fidell,
2001) is strongly discouraged. Removing influential outliers before recognizing
that they are good cases will prematurely worsen model fit whereas removing
non-influential outlying cases will only reduce sample size and thus statistical
power and precision of parameter estimates. Additionally, screening data using

information based on outlying status does not adequately address the presence of
influential non-outliers. Sensitivity analysis in SEM should therefore be focused
on identifying influential cases instead of outliers.
The issue of sample size impacts the analysis of case influence in at least
two ways. The first involves the computation burden. Case influence quantified
by deletion statistics has a clean interpretation based on empirical results in
that they measure the effect of removing case i from the sample. However, the
computational cost of obtaining these global estimates increases with sample
size as N sets of model results need to be computed from the N delete-i

samples, with each set of results then compared with results obtained from the
full sample. Alternatively, local measures of case influence based on influence
curves (Cadigan, 1995; Cook, 1986; Tanaka, Watadani, & Moon, 1991) and
the concept of person fit (Coffman & Millsap, 2006; Reise & Widaman, 1999)
may be obtained from solutions obtained from the full sample only. At best,
these local measures of influence closely approximate the effects of removing
case i from the analysis. Yet, there have been instances where such local
approximations grossly misrepresent the empirical effect of removing case i
from the sample (e.g., Pack, Jolliffe, & Morgan, 1988).
A second impact of sample size on the analysis of case influence is that the
degree of influence that cases may exert on results is moderated by sample size.
Indeed, the magnitude of case influence measures is weighted by sample size in
that cases belonging to smaller samples would tend to exhibit relatively more
influence compared to cases belonging to larger samples. At exceedingly large
sample sizes, it is unlikely for a single case to have strong influence on results
as its contribution in terms of data is minimal. At small sample sizes cases
may exert much more influence as their data represent a large proportion of the
data. Methods described in this article will be more informative and useful in
applications of SEM where sample size is not extremely large.
Sensitivity analysis of cases in SEM is desirable in practice as it enhances
the quality of results by ascertaining their robustness through the identification
of unusual cases that have undue influence on estimates. A finding of the
absence of cases that have high influence on key results is most desirable, of
course. But the identification of highly influential cases provides the investigator
with information that can be used to modify analyses or at least gain a richer
understanding of results. Given that the fundamental goal of science is to
comprehend nature in the most parsimonious manner, scientists are charged
with developing and validating models that capture major regularities in data
and are robust to noise. Information gained from the study of case influence in
SEM helps to correct for execution or coding errors (Barnett & Lewis, 1994)
and to avoid drawing conclusions driven by only a few cases in a sample.
The benefits of investigating case influence are widely recognized in regression
analysis. Sensitivity analysis of cases in SEM should be similarly encouraged

and implemented.
ACKNOWLEDGMENTS
We thank Li Cai for sharing his program that estimated initial models. We thank
Kenneth A. Bollen for generously sharing data for the empirical example on the
general structural equation model.
REFERENCES
Barnett V., & Lewis, T. (1994). Outliers in statistical data (3rd ed.). New York, NY: Wiley.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality
and Social Psychology, 51, 1173–1182.
Belsley, K. A., Kuh, E., & Welsch, R. E. (1980). Regression diagnostics: Identifying influential data
and sources of collinearity. New York, NY: Wiley.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107,
238–246.
Bollen, K. A. (1987). Outliers and improper solutions: A confirmatory factor analysis example.
Sociological Methods and Research, 15, 375–384.
Bollen, K. A. (1989). Structural equation models with latent variables. New York, NY: Wiley.
Bollen, K. A., & Arminger, G. (1991). Observational residuals in factor analysis and structural
equation models. Sociological Methodology, 21, 235–262.
Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance struc-
tures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.
Browne, M. W., & Arminger, G. (1995). Specification and estimation of mean- and covariance
structure models. In G. Arminger, C. Clogg, & M. Sobel (Eds.), Handbook of modeling in the
social and behavioral sciences (pp. 185–249). New York, NY: Plenum Press.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, &
S. Long (Eds.), Testing structural equation models (pp. 131–161). Newbury Park, CA: Sage.
Cadigan, N. G. (1995). Local influence in structural equation models. Structural Equation Modeling,
2, 13–20.
Coffman, D. L., & Millsap, R. E. (2006). Evaluating latent growth curve models using individual
fit statistics. Structural Equation Modeling, 13, 1–27.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation
analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
Cook, R. D. (1977). Detection of influential observations in linear regression. Technometrics, 19,
15–18.
Cook, R. D. (1986). Assessment of local influence. Journal of the Royal Statistical Society B, 48(2),
133–169.
Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. New York, NY: Chapman
& Hall.
Farman, J. C., Gardiner, B. G., & Shanklin, J. D. (1985). Large losses of total ozone in Antarctica
reveal seasonal ClOx/NOx interaction. Nature, 315, 207–210.
Hawkins, D. M. (1980). Identification of outliers. London, UK: Chapman & Hall.
Holzinger, K., & Swineford, F. (1939). A study in factor analysis: The stability of a bi-factor solution.
Chicago, IL: University of Chicago.
Lee, S. Y, & Wang, S. J. (1996). Sensitivity analysis of structural equation models. Psychometrika,
61, 93–108.
Mahalanobis, P. C. (1936). On the generalized distance in statistics. Proceedings of the National
Institute of Science of India, 2, 49–55.
Morgan, T. H. (1910). Sex limited inheritance in drosophila. Science, 32, 120–122.
Pack, P., Jolliffe, I. T., & Morgan, B. J. T. (1988). Influential observations in principal component
analysis: A case study. Journal of Applied Statistics, 15, 37–50.
Rasmussen, J. L. (1988). Evaluating outlier identification tests: Mahalanobis D Squared and Comrey
D. Multivariate Behavioral Research, 23, 189–202.

Reise, S. P., & Widaman, K. F. (1999). Assessing the fit of measurement models at the individual
level: A comparison of item response theory and covariance structure approaches. Psychological
Methods, 4, 3–21.
Rousseeuw, P. J., & Van Zomeren, B. C. (1990). Unmasking multivariate outliers and leverage
points. Journal of the American Statistical Association, 85, 633–651.
Sobel, M. E. (1982). Asymptotic intervals for indirect effects in structural equations models. In S.
Leinhart (Ed.), Sociological methodology 1982 (pp. 290–312). San Francisco, CA: Jossey-Bass.
Steiger, J. H., & Lind, J. C. (1980, May). Statistically-based tests for the number of common factors.
Paper presented at the annual Spring meeting of the Psychometric Society, Iowa City, IA.
Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights,
MA: Allyn & Bacon.
Tanaka, Y., Watadani, S., & Moon, S. H. (1991). Influence in covariance structure analysis: With an
application to confirmatory factor analysis. Communications in Statistics, Series A, 20, 3805–3821.
Thurstone, L. L. (1938). Primary mental abilities. Chicago, IL: University of Chicago Press.
Traxler, A. E. (1934). Traxler silent reading test, form 1, for grades 7 to 10. Bloomington, IL: Public
School Publishing Company.
Tucker, L. R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis.
Psychometrika, 38, 1–10.
Yuan, K., & Bentler, P. M. (2001). Effect of outliers on estimators and tests in covariance structure
analysis. British Journal of Mathematical and Statistical Psychology, 54, 161–175.

Pek 2011

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pek 2011

Uploaded by

Copyright:

Available Formats

This article was downloaded by: [Central Michigan University]

On: 19 December 2014, At: 14:34

To link to this article: http://dx.doi.org/10.1080/00273171.2011.561068

PLEASE SCROLL DOWN FOR ARTICLE

Cases and Their Influence

Jolynn Pek and Robert C. MacCallum

The detection of outliers and influential observations is routine practice in linear

Mathematical models empower researchers with the ability to summarize, un-

Correspondence concerning this article should be addressed to Jolynn Pek, Department of

approximations of complex mechanisms and cannot be viewed as exact represen-

capture major regularities in the data while remaining relatively insensitive to

also be subject to exploratory study. The observation of three white-eyed flies

preventing the unhappy circumstance wherein important aspects of results and

CASE INFLUENCE IN STRUCTURAL

SEM is a general modeling framework where models of linear relations among

Overall Model Fit

where ™O and ™O .i / are the k 1 vectors of estimated model parameters based

of displacement in the likelihood function caused by removing case i from the

FML D ln j†j ln jSj C trŒS† 1  p; (2)

¦2i D ¦2 ¦2.i / ; (4)

where F .:/ is the ML discrepancy function minimized to obtain estimates of ™O

Case influence on a vector of parameters may be quantified by the generalized

gCDi D .™O ™O .i //0 VAR.

where ™O and ™O .i / are l 1 vectors of parameter estimates obtained from the

gCDi D Œ™O j i 2 ; (8)

when case influence on a single parameter estimate j is assessed using gCD.

ML and asymptotically distribution free estimation (Browne, 1984) leading to

MDi D .xi x/0 S 1 .xi x/; (9)

where xi is the p 1 vector of p observed variables for case i , x is the p 1

COVRATIO D jS.i /j=jSj; (10)

Si D ln jS.i /j ln jSj C t r .S.i / S 1 / p; (11)

Influential Cases Versus Outliers

TWO EMPIRICAL APPLICATIONS

To illustrate and highlight several issues involved in interpreting measures of case

Six Psychological Tests

where † is the p p population covariance matrix, ‰ is the m m correlation

‚ is constrained to be diagonal, reflecting the assumption that the observed

91 1.432 1.420 1.719 0.817 1.039 0.904 16.268

96 1.732 1.200 0.120 2.711 1.264 2.105 17.932

Parameter Estimate Estimate

œ1;1 0.587 0.539

œ3;1 0.890 0.969

Note. Model A D restricted factor analysis with correlated factors.

Model B .¦2B D 4:003/. The extent and direction of influence of a case on

The direction of influence a case exerts on overall model fit distinguishes

FIGURE 3 Case influence on a vector of parameter estimates (Generalized Cook’s

open circles in Figure 3) without regard to the number of parameters, we

Across Figures 1 through 4, potential outliers (located to the right of all

Industrialization and Democracy

alization in 1960. Industrialization in 1960 was indicated by three measured

is specified to contemporaneously influence democracy in 1960 .˜2 / and in 1965

Further study of the results in Figure 7 reveals an interesting phenomenon

where “O 2;1 is the effect of industrialization on democracy in 1960 and “O 3;2

“O 2;1 “O 3;2 “O 2;1.i / “O 3;2.i /

DISCUSSION AND RECOMMENDATIONS

power and precision of parameter estimates. Additionally, screening data using

size as N sets of model results need to be computed from the N delete-i

analysis. Sensitivity analysis of cases in SEM should be similarly encouraged

general structural equation model.

D. Multivariate Behavioral Research, 23, 189–202.

You might also like

FML D ln j†j ln jSj C trŒS† 1 p; (2)

¦2i D ¦2 ¦2.i / ; (4)

gCDi D Œ™O j i 2 ; (8)

Si D ln jS.i /j ln jSj C t r .S.i / S 1 / p; (11)

Model B .¦2B D 4:003/. The extent and direction of influence of a case on