Chapter 35
Item-Level Factor Analysis
Brian D. Stucky, Nisha C. Gottfredson, and A. T. Panter
Copyright American Psychological Association. Not for further distribution.
Fundamental research questions in psychology center
on establishing the factor structure of new measures
and understanding how unobserved constructs
assessed by these measures relate to other constructs.
The standard factor analytic model assumes that
response data are based on continuously measured
indicators of constructs, yet the individual items used
in social science research rarely meet this assumption.
Items appearing in surveys (such as in attitude or
knowledge scales), obtained through observation
codes, or gathered in interview settings may be
dichotomous, ordered categorical, censored counts or
zero-inflated counts. They may also have a variety of
other distribution forms. This chapter addresses the
historical traditions and current data analytic recom-
mendations for conducting data analysis using latent
variable models with item-level indicators.
The general topic of how to describe the factor
structure and item-level characteristics of a collec-
tion of items emerged from two main theoretical
routes (social sciences: item-level factor analysis
[IFA]; education: item response theory [IRT]); the
connection between these routes has been eluci-
dated formally in the past few decades. In consider-
ing analytic options for modeling item-level data, we
first discuss issues related to item measurement,
response distributions, and estimation methods. We
then address appropriate modeling conditions for
focusing on individual items versus composites that
are formed by the researcher from item subsets,
such as item parcels or testlets, and we review
The authors thank members of the PROMIS pediatric network (funded by Grant 5U01AR052181) and Deb Irwin, Michelle Langer, David Thissen, Esi
DeWitt, Jim Varni, Karin Yeatts, and Darren DeWalt for use of these data.
DOI: 10.1037/13619-036
APA Handbook of Research Methods in Psychology: Vol. 1. Foundations, Planning, Measures, and Psychometrics, H. Cooper (Editor-in-Chief)
Copyright © 2012 by the American Psychological Association. All rights reserved.
methods for assessing measurement invariance
across groups. Because most commercially available
factor analysis and structural equation modeling
(SEM) computer programs allow for estimation
approaches and options for conducting item-level
analyses, distinctions among computer programs are
made when useful. Finally, specific recommenda-
tions are provided about appropriate research design
conditions for modeling item-level data.
Factor Analysis
Factor analyses of categorical and continuous mea-
sures have the same goal: to explain the relation-
ships among the variables by a smaller set of
underlying, or latent, variables. Traditionally, factor
analysis emphasizes understanding the dimensional-
ity of a set of variables (e.g., exploratory factor anal-
ysis [EFA] and confirmatory factor analysis [CFA],
respectively) and modeling the causal relationships
among those dimensions (e.g., SEM). Although
these analytic procedures are applicable for categori-
cal response data, the key difference between the
approaches lies in where the investigator places
emphasis and the nature of the research questions.
In the historical treatment of factor analysis for con-
tinuously measured items, item-level characteristics
were often seen as problematic (McDonald, 1967,
1999; McDonald & Ahlawat, 1974). For example,
the very notion of item difficulty (the probability of
endorsement) is conceptually challenging from a
683
 Stucky, Gottfredson, and Panter
continuously distributed factor analytic perspective,
and hence techniques such as data transformations
and standardization are commonly used to mask
item differences in response distributions and scales.
The IFA approach, on the other hand, appreci-
ates item behavior and characteristics as intrinsically
meaningful. Concepts such as item discrimination
(the strength of the relationship between an item
and its factor) and item difficulty are key tools that
psychometricians use to analyze item response data.
To further develop the importance of item charac-
teristics, we provide some background on the tradi-
tional factor analysis model.
Copyright American Psychological Association. Not for further distribution.
The common factor model (Jöreskog, 1969,
1970) is based on a set of assumptions that the
observed variables y = (y1, . . ., yi) are continuous,
normally distributed with mean σ and covariance Σ
and that the relationship between observed and
latent variables ξ = (ξ 1,. . ., ξ p) is best described by
a linear model:
y =  +  + e , (1)
where observed responses y are functions of factor
loadings Λ, latent variables ξ, item means α, and
measurement error (residuals). Incorporating the
covariance matrix of ξ, called Φ, and the covariance
matrix of (typically uncorrelated) residuals Ψ, the
relationships among y are represented by the covari-
ance matrix Σ:
 = ′ +  . (2)
Items appearing in research in psychology and
the behavioral sciences typically have categorical
response options that violate these assumptions
(Krosnick, 1999). In these situations, an appropriate
alternative to the linear common factor model is IFA
(Mislevy, 1986; Muthén, 1978; Wirth & Edwards,
2007). Categorical confirmatory factor analysis
(CCFA) assumes that a researcher is focused on the
measurement features of responses to categorical
items and that these items are “discrete representa-
tions of continuous latent responses” (Wirth &
Edwards, 2007, p. 59). Items with these characteris-
tics range from dichotomous responses common in
education settings (i.e., correct or incorrect, true or
false) to polytomous responses, or ordered categori-
cal responses, now common in most psychological
684
disciplines (i.e., Likert-type scales such as agree-
ment, self-description).
It is not statistically appropriate to ignore the cat-
egorical nature of items and apply the common fac-
tor model to such data; attempting to do so can
result in a variety of problems. Perhaps most central,
categorical response variables are bounded by the
minimum and maximum response options (e.g., 0
for an item that is not endorsed and 1 for an item
that is endorsed), and only through nonlinear func-
tions (such as the normal ogive and logistic cumula-
tive distribution functions) is one able to place
discrete item responses on the scale of the unob-
served continuous latent variable (McDonald &
Ahlawat, 1974). If ignored, the model is misspeci-
fied, and the linear approximation of the nonlinear
relationship will be heavily influenced by the
observed variable’s mean (McDonald, 1999; Mislevy,
1986). Second, fitting a factor analysis linear model
to categorical data often leads to biased parameter
estimates (DiStefano, 2002). In addition, factor load-
ings may be attenuated in cases of sparse response
coverage (such as skewness), which is typical of
scale items with response options that measure only
extreme trait locations. The accumulation of these
problems results in untrustworthy model fit
statistics.
In parallel with emergent developments for the
factor analysis of dichotomous and ordered categori-
cal data in the mid-1980s, important advances in the
IRT tradition also allowed for analyses of these types
of items (Mislevy, 1986). Examples from early
adopters of these two traditions in psychology and
related fields include assessment of depression
(Childs, Dahlstrom, Kemp, & Panter, 1992; Schaef-
fer, 1988), need for cognition (Tanaka, Panter, &
Winborne, 1988), self-monitoring (Panter &
Tanaka, 1987), and psychopathology and personal-
ity (e.g., Reise & Waller, 1990) as well as several
studies focusing on general issues in item order,
scale design, and administration (Panter, Tanaka, &
Wellens, 1992; Steinberg, 1994; Waller & Reise,
1989). Review articles and early conference presen-
tations were specifically targeted at communicating
these developments in the analysis item response
data to psychological researchers (e.g., Panter,
Swygert, Dahlstrom, & Tanaka, 1997; Reise,

Widaman, & Pugh, 1993; Steinberg & Thissen,
1995; Thissen, 1992).
Thissen and Steinberg (1986) described a taxon-
omy of models in the psychometric literature to
properly analyze categorical response data in a con-
firmatory analytic setting allowing items to be linked
to underlying latent variables. Many of these models
originated in the IRT tradition, including one-, two-,
and three-parameter logistic models for binary data,
the graded-response model (GRM; Samejima, 1969),
the generalized partial credit model (Muraki, 1992)
for ordered categorical response items, models for
nominal response data (Bock, 1972), and many oth-
Copyright American Psychological Association. Not for further distribution.
ers. For simplicity of discussion, however, we
review the binary–dichotomous item response case
(e.g., agree or disagree) as well as a generalization to
model polytomous item responses (e.g., Likert-type
items).
For dichotomous items, a respondent’s location
on the continuous underlying trait (ξ in factor anal-
ysis notation and θ in IRT notation) is expressed
through her choice of response option. In other
*
words, one’s location on the latent variable, yi , is
modeled through the common factor and item-
specific residual:
y*i = i  + e i , (3)
*
where the continuous latent response yi has a vari-
ance of 1.0. Assuming that higher scores reflect
more of a given trait, the relationship between
*
observed item responses yi, location on the trait yi ,
and the threshold parameters is
yi = 1 if y*i ≥ oi
(4)
0 if y*i ≥ oi
The answers that respondents provide to a given
scale item can be arranged along the latent response
*
distribution yi , by thresholds, τi. Thus, a threshold
represents the location on the latent continuum that
separates discrete responses to a given item.
The Item Response Theory Tradition
Chapter 36 of this volume reviews traditional IRT
modeling and several new directions in the field. We
discuss the more common set of IRT models, their
Item-Level Factor Analysis
assumptions, and their uses to motivate our discus-
sion of the close relation between IFA and IRT.
When binary data are modeled in an IRT context,
one often uses logistic functions. While capitalizing
on the mathematical convenience of the model
(Birnbaum, 1968, p. 400), the logistic function
closely follows the cumulative normal distribution.
The two-parameter logistic (2PL) model may be
written as
 1 
T  ui = 1|  = . (5)
 (1 + exp[− Da i ( − bi )]

In keeping with Lazarsfeld’s (1950) classic nota-
tion, T traces the probability of a correct response,
u = 1, to item i conditional on ability θ, whereas an
incorrect response has the probability T(0|θ) = 1 −
T(1| θ), or T0 = 1 − T1(θ) in more compact notation.
The logistic function describes each response proba-
bility on the basis of the slope or discrimination
parameter (ai), which indicates the strength of asso-
ciation between the item response and latent dimen-
sion, and a difficulty or threshold parameter (bi),
which indicates the location on the ability contin-
uum for which individuals’ probability of correct
response is 50%. The constant D (approximately
1.7) is commonly used to place the model on the
same scale as the normal ogive, the precursor to the
logistic representation (Lord, 1952).
For polytomous (ordered categorical) response
data, a generalization of the 2PL, Samejima’s (1969)
GRM, is often used. The GRM describes the proba-
bility of a response in category k or higher, where
k = 0, 1, . . ., m − 1:
 
1
T *  yi = k |  = , (6)
 (1 + exp[−a i ( − bik )]

noting that T*(0|θ) = 1 and T*(m|θ) = 0. Here the
probability of responding in category k is the differ-
ence between the probabilities of responding in k or
higher and the higher response:
Ti (k | ) = Ti* (k | ) − Ti* (k +1 | ). (7)
The model describes the response process by
estimating one less threshold than the number of
response alternatives. Each threshold or cutpoint
685
 Stucky, Gottfredson, and Panter
marks the boundary between a response in category
k from k + 1. Although such a model is useful for a
variety of psychological scales with ordinal response
category items, many other IRT models characterize
item responses (for a review, see Embretson &
Reise, 2000).
Relations Between the Two Traditions
The histories and notational differences between
Jöreskog’s (1969) common factor model and the IRT
models popularized by Lord and Novick (1968)
seem to indicate some inherent difference across the
traditions. Actually, the relation between IRT and
Copyright American Psychological Association. Not for further distribution.
CCFA frameworks has been presented by Bartho-
lomew (1983), Muthén (1983), and Muthén and
Lehman (1985). Takane and de Leeuw (1987) have
provided the mathematical equivalence between the
common models. Indeed, under certain conditions,
IFA can be conducted entirely in an IRT framework.
Given item parameters in either IRT or IFA, one
may translate and back-translate between parame-
terizations. IRT-equivalent slopes and thresholds
may be obtained easily from factor analysis notation:
 a 
ai =  i  D and bi = a i ,
 1+ a  y*i
 i  
where a and b are as previously defined, and τi rep-
resents the location on the latent variable scale that
distinguishes between responses. The square root of
one minus the squared factor loading is the residual
* *
standard deviation of yi (with yi standardized to
have a variance of 1.0, as is common in many IFA
approaches). Additionally, if the IRT parameters are
in the normal metric, the scaling factor D is not
needed and drops out of the equation. With little
algebra, one may translate the IRT parameters back
into factor analysis notation loadings (Λ) and
thresholds (τ):
   
ai / D  and  =  (a i / D )bi
i =   . (9)
 1 + (a / D )2  i
 1 + (a / D )2  (a) diagonally weighted least squares (Jöreskog &
 i   i  
Actually, most software programs estimate the
model in a slope-intercept parameterization. This
parameterization utilizes the logit, Dai (θ – bi), and
after multiplying the slope through, provides the
686
intercept, –aibi, which, when D = 1 is the log odds of
a correct response at θ = 0. In typical IRT applica-
tions, however, intercepts are converted into the
common slope-threshold parameterization by divid-
ing the intercept by the negative slope.
As Wirth and Edwards (2007) made clear, there
is some practical utility in these conversions. In
many instances these translations help identify
unreasonable parameter estimates. Item-level data
are virtually never perfectly related to the latent
trait. More often, items contain some degree of error
variance, so when this error variance becomes suspi-
ciously low, one can anticipate when item parame-
ters are untrustworthy. Wirth and Edwards (2007)
suggested skepticism when interpreting factor load-
ings > .95 (and hence a parameters above about 3)
and objection to loadings > .97 (and a parameters
above about 4). In both instances, such values would
be considered near-Heywood (1931) cases.
Estimating Models Within the
Two Traditions
If one considers IFA to be the intersection of tradi-
tional factor analysis and IRT, then it may be helpful
(8)
to consider why these closely related techniques
evolved from largely separate literatures. In part, the
distinction has less to do with differences in the
hypothesized models for item responses and more to
do with differences in estimation employed within
each tradition (Mislevy, 1986). Estimation methods
for CCFA traditionally made use of the sample tetra-
choric (for binary–dichotomous data) and poly-
choric (for polytomous–ordered categorical data)
correlation matrix (Christofferson, 1975; Muthén,
1978, 1984) in addition to a weight matrix that grew
substantially with the number of items. Mixtures of
item types, which often occur in research settings in
psychology, also could be handled as well as models
with many factors (e.g., SEM). Two widely used
least squares estimators that make use of the sam-
ple tetrachoric–polychoric correlation matrix are
Sörbom, 2001) as used in LISREL (Jöreskog &
Sörbom, 2001), and (b) mean- and variance-
adjusted weighted least squares (WLSMV; Muthén,
du Toit, & Spisic, 1997) as used in Mplus (Muthén &
Muthén, 1998–2007). Both estimators use only the

diagonal elements of a weight matrix (see Jöreskog,
1990; Muthén, 1984), thereby greatly reducing the
burdensome operation of inverting a weight matrix
for models with many items (for some other estima-
tors see Christofferson, 1975; Jöreskog & Sörbom,
2001).
An alternative to weighted least squares estima-
tion is full-information maximum likelihood
(FIML), traditionally used in IFA and SEM
approaches. Rather than employing a polychoric
correlation matrix, FIML takes full advantage of the
entire data matrix. Here, the underlying data struc-
ture is reproduced by way of the matrix of factor
Copyright American Psychological Association. Not for further distribution.
loadings (Λy), covariance matrixes of latent vari-
ables (Φξ), and measurement errors (Θδ). The FIML
method produces unbiased standard errors for the
model parameters (factor loadings, interrelations
among latent variables, measurement errors) and
model fit indexes, unlike adjusted weighted least
squares approaches that require corrections for
biased standard errors and fit indexes (see Satorra &
Bentler, 1994). FIML introduces the problem
of multiple dimensions, however. Specifically,
integration—finding the area of a region defined by
a function—must be approximated by a number of
quadrature points (e.g., Gauss-Hermite) over the
number of dimensions. So, for many reasonable-size
models (and quadrature points), computing time is
calculated in hours, a problem less often encoun-
tered when using least squares estimators.
Alternatively, estimation of IRT parameters is
typically conducted using maximum likelihood.
Birnbaum’s (1968, p. 420) use of joint-maximum
likelihood (JML) provided the first directly estimated
IRT parameters.1 With JML, the parameters describ-
ing how the item behaves and the score reflecting a
person’s level on the underlying trait dimension are
simultaneously estimated. Currently, the popularity
of JML has been surpassed by the use of marginal
maximum likelihood (MML; Bock & Lieberman,
1970), which, rather than attempting to estimate
item and person parameters simultaneously, consid-
ers person parameters as nuisance parameters
(and integrates over them), leaving only the item
Rasch (1960) provided a substantially simpler estimation method, conditional maximum likelihood (CML), which capitalizes on the fact that summed
1
item scores are sufficient statistics. CML estimates item difficulty (the only item parameter estimated; discrimination parameters are 1.0) conditional
on summed scores.
Item-Level Factor Analysis
parameters to be estimated in the marginal distribu-
tion. Application of the expectation–maximization
(EM) algorithm (Bock & Aitkin, 1981) alleviated
much computational time for models with few fac-
tors and many items by iteratively estimating trial
item parameters, then using these to find the
expected number of responses and the proportions
of individuals at given levels of the latent variable
(E-step), and finally resubstituting these values back
into the likelihood equation (M-step). MML/EM is
used in widely available IRT software, such as
BILOG-MG (Zimowski, Muraki, Mislevy, & Bock,
2003), MULTILOG (Thissen, 2003), and soon,
IRTPRO (Cai, du Toit, & Thissen, 2011).
More recently, advances in parameter estimation
have overcome the challenge of dimensionality
posed by using MML/EM, and have made high-
dimensional IRT and IFA models more tractable
(e.g., multidimensional IRT [MIRT]; Reckase, 2009).
That is, when researchers discretely measure items,
they can now model a larger number of underlying
factors than was previously possible in traditional
unidimensional IRT models. Here, we briefly high-
light three recent developments that, with time, may
see more use from applied researchers in psychology
and the behavioral sciences with high-dimensional
data: (a) EM with adaptive quadrature (ADQ),
(b) Markov chain Monte Carlo (MCMC), and
(c) the Metropolis-Hastings Robbins-Monro
(MH-RM) algorithm for maximum likelihood esti-
mates (MLEs). For a moderate number of dimen-
sions, ADQ is an attractive alternative to MML/EM.
Instead of using fixed-point quadrature, ADQ adapts
the number of points needed so that the estimation
process becomes more efficient (Schilling & Bock,
2005). More recently, MH-RM (Cai, 2010a) uses a
Metropolis-Hastings (Hastings, 1970) Robbins-
Monro algorithm (Robbins & Monro, 1951), which
has enabled efficient estimation of high-dimensional
models that had previously been intractable.
(MH-RM will be an estimation option in the soft-
ware package IRTPRO; Cai et al., 2011). Finally,
MCMC algorithms allow for the inspection of the
quality of other maximum likelihood estimates by
687
 Stucky, Gottfredson, and Panter
constructing a Markov chain with a stationary distri-
bution as its target, at which point samples (i.e.,
random draws) taken from the chain serve as poste-
rior estimates. Although MCMC is an attractive
solution to dimensionality problems in FIML
contexts, the efficiency of current MCMC tech-
niques (i.e., the computational burden; Edwards,
2010) makes its use, from an applied researcher’s
perspective, somewhat limited. Hopefully, however,
these new estimation approaches will provide
psychologists with the tools to test models whose
complexity exceeded the boundaries of the past
software.
Copyright American Psychological Association. Not for further distribution.
The preceding discussion highlighted two ana-
lytic traditions that focus on understanding the
underlying factor structure of item-level data with
different measurement levels. The less than continu-
ously measured items present more of an analytic
challenge than do the continuously measured indi-
cators. In the next section, we consider current
research on a related approach that investigators
have used to circumvent complexities associated
with analyses conducted at the item level.
If Items Are More Complicated to
Analyze, Why Not Parcel Them?
Under certain circumstances, a researcher may
decide to aggregate (i.e., create a sum score from) a
collection of items to serve as factor indicators,
rather than using individual items as indicators,
before conducting their factor analysis. This practice
is known as parceling.2 Parceling involves splitting a
relatively large number of items thought to repre-
sent a latent variable into a smaller set of sum
scores. These sum scores are then used in place of
the individual items to identify the latent variable.
Little, Cunningham, Shahar, and Widaman
(2002) and Little, Lindenberger, and Nesselroade
(1999) described a latent variable’s domain as con-
sisting of an infinite pool of potential items, each
deviating to some degree from the centroid of the
latent variable. Assuming that the expected value of
these deviations is zero, and assuming that each
In IRT, aggregated indicators of latent variables are called testlets. In this setting, testlets are formed to alleviate local dependence between correlated
2
items that is irrelevant to the latent variable of interest, often resulting from related test sections or similar item stems (Wainer, Bradlow, & Wang,
2007).
688
item’s error is uncorrelated with every other item’s
error, then a parcel that is constructed of a randomly
selected sample of items should be, on average, a
less biased and more reliable indicator of the latent
construct. If, however, items are not conditionally
independent from one another after accounting for
the common factor (i.e., item responses are related
for a reason other than the underlying dimension
such as when two or three items share similar word
stems), then parcels of these correlated items will be
biased away from the latent variable’s centroid. If it
makes sense to conceive of sampling from an infi-
nite pool of independently distributed items in a
given content domain, then using parcels will repre-
sent the latent construct in a more stable, replicable
way than if the same number of items were used to
represent the same latent construct. Indeed, a fre-
quently cited benefit of parceling is improved indi-
cator reliability (Cattell & Burdsal, 1975). The
Spearman–Brown prophecy formula reveals that if
each item representing a latent construct consists
partially of true score variability and partially of
error variance, then the sum of several such items
will contain a higher proportion of true score vari-
ability than any individual item (Coffman &
MacCallum, 2005).
Coffman and MacCallum (2005) advocated using
parcels as latent variable indicators when such a
large number of items are needed to achieve ade-
quate representation of the latent variable domain
and when it is implausible to use individual items as
factor indicators. For example, if a researcher has a
very large number of items to factor analyze, IFA
estimation in both of the traditions is more difficult.
In such situations, when there are too many items to
estimate a latent variable using individual items,
researchers are forced to choose between (a) creat-
ing parcels so that model estimation is possible
(because estimation with many discrete items is
computationally intensive) or (b) aggregating the
items into a single measured variable (e.g., by
summing the items or outputting an estimated fac-
tor score). Coffman and MacCallum demonstrated
that the parceling method is superior to alternative
 Item-Level Factor Analysis

analysis options, such as path analysis, that do not
allow the explicit modeling of unreliability of
measurement.
Especially relevant to IFA, many researchers who
use parcels do so to meet normality assumptions of
their estimation method. When items are Poisson-
distributed counts, dichotomous, or ordinal with
only a few response categories (e.g., a 4-point
Likert-type scale), then individual item distributions
will badly violate normality assumptions. If normal-
ity is assumed, parameters obtained by maximum
likelihood (ML) or generalized least squares (GLS)
estimation will be downwardly biased. In other
Copyright American Psychological Association. Not for further distribution.
with the Satorra–Bentler (1994) correction. The
authors did not find support for the claim that factor
loading estimates would be less biased when parcels
are used, but they did find that estimates were less
variable with this technique. The Satorra–Bentler
correction resulted in less biased parameter esti-
mates. Bandalos (2008) compared the parceling
strategy to WLSMV estimation with nonnormal
items and found that parameter estimates were
biased when parcels were used, particularly when
unidimensionality of parcels was violated. This
result was obtained for both factor loadings and
structural parameter estimates between latent

words, using this approach will lead to serious con-
sequences: Standard errors for factor loadings will
be underestimated and chi-square tests of model fit
will be too high (West, Finch, & Curran, 1995).
West et al. (1995) suggested three potential solu-
tions for handling nonnormal factor indicators.
First, they suggest that researchers can use the
Satorra–Bentler correction for nonnormal data in
conjunction with the ML estimator for a better
approximation of the model chi-square statistic, fac-
tor loadings, and associated standard errors (Satorra
& Bentler, 1994). Second, it may be feasible to use
an alternative estimator that does not require items
to be normally distributed, such WLSMV. Flora and
Curran (2004) found that WLSMV works well even
if the latent distribution underlying the discretely
distributed observed variable is not normally distrib-
uted. Finally, West et al. (1995) suggested that a
researcher may create parcels so that factor indica-
tors more closely approximate a normal distribution
to use with ML or GLS.
Hau and Marsh (2004) evaluated the use of par-
cels as a technique for handling nonnormally dis-
tributed items when the WLSMV assumption of
continuous latent underlying variables is not plausi-
ble. They compared this method to the technique of
using nonnormally distributed items as indicators
factors. Furthermore, Bandalos showed that model
fit was overestimated (i.e., thought to be better
than it was) if parcels were used when unidimen-
sionality was violated. In contrast, estimates
obtained using WLSMV estimation were unbiased,
particularly when sample size increased.
Several authors have shown that parceling leads
to overly optimistic model fit indexes when multidi-
mensionality is present (Bandalos, 2008; Little et al.,
2002). When locally dependent items are combined
into a parcel, the irrelevant item correlation is attrib-
uted to shared variance because of the common fac-
tor, thus masking multidimensionality and
artificially inflating model fit. Bandalos pointed out
that unplanned multidimensionality is common;
method factors are one example of this. In addition
to inflating model fit, the masking of a multidimen-
sional factor structure leads to confounded and
uninterpretable latent constructs (Hagtvet & Nasser,
2004). Given the potential pitfalls of parceling with
locally dependent items, Coffman and MacCallum
(2005) and Little et al. (2002) recommended testing
the unidimensionality assumption and proceeding
with parceling only if the assumption is reasonable.3
When researchers decide whether they should
factor analyze individual items that are not continu-
ously measured, the current research suggests that

If violations of unidimensionality exist, the joint probability of item responses is no longer equal to the product of marginal probabilities, which leads
3

to biased parameter estimates (Reckase, 1979) and overestimates of score precision (Thissen, Steinberg, & Mooney, 1989). Methods for detecting
violations of unidimensionality may be broadly categorized as those stemming from unidimensional models that are diagnostic tests (e.g., Chen &
Thissen, 1997; Yen, 1984), those that introduce additional model parameters and latent variables to account for local dependence (e.g., Bradlow,
Wainer, & Wang, 1999; Hoskens & De Boeck, 1997), and those that test the assumption of conditional independence (e.g., Stout, 1987). Diagnostic
methods are useful data analytic tools for researchers interested in identifying locally dependent pairs or subsets of items. Modeling the local depen-
dence directly is useful in situations in which local dependence is expected and is a requirement of the test (e.g., the use of testlets in modeling
passage-dependent items; Wainer & Kiely, 1987).

689
 Stucky, Gottfredson, and Panter
parcels are more reliable than individual items and
may allow researchers to represent more fully the
domains of their latent constructs. Parcels seem to
be the best option when too many items are present
to estimate a full measurement model. On the other
hand, if the relation between items and factors is
substantively meaningful, parcels may mask true
relations that exist within the data, such as local
dependence or a complex factor structure (Little
et al., 2002). Furthermore, alternative methods of
estimation exist that negate the necessity of parcel-
ing in most circumstances, and these alternatives
have been empirically demonstrated to produce less
Copyright American Psychological Association. Not for further distribution.
biased estimates than those obtained with parceling.
Given these contrasting arguments, Little et al.
(2002) suggested using parcels when unidimension-
ality is ensured, when constructs have been well
established and defined, and when the measurement
model is not of primary interest. That is, parceling
may be a reasonable technique to use when the
structural relations among latent variables are more
interesting than the measurement model. Alterna-
tive methods, such as analyses from the factor ana-
lytic tradition, should be considered before
proceeding with a parceling strategy.
Measurement Invariance
Complexities in Item Factor
Analysis
Testing whether the factor structure of items is simi-
lar or invariant across independent groups is an
important extension of the IFA problem.4 Inferences
about group differences are only accurate if the
latent construct being measured is invariant across
groups.
Types of Invariance
Configural invariance occurs when the same factor
structure exists across groups (Thurstone, 1947). If
factor loadings are equivalent across groups such
that a one-unit increase in the latent variable mean
is associated with an identical λ unit increase in the
The factor analytic idea of measurement invariance parallels the concept of differential item functioning (DIF) within IRT. DIF exists when, controlling
4
for individuals’ true latent variable score, the conditional probability of answering an item correctly is not the same across groups of individuals (i.e.,
measurement noninvariance). This is a major concern for psychometricians who are responsible for creating bias-free standardized tests.
690
expected value for all items, regardless of group
membership, then weak factorial invariance exists
(Horn & McArdle, 1992; Millsap, 1997; Millsap &
Kwok, 2004; Millsap & Meredith, 2007; Thurstone,
1947; Widaman & Reise, 1997). If weak factorial
invariance is met and item intercepts and thresholds
are constant across groups, then there is strong facto-
rial invariance (Meredith, 1993; Millsap & Kwok,
2004; Millsap & Meredith, 2007; Steenkamp &
Baumgartner, 1998). Strong factorial invariance is
desirable because it means that no systematic differ-
ences are present in the measurement models across
groups. Partial invariance occurs when some, but
not all, factor loadings, means, and thresholds are
invariant (Millsap & Kwok, 2004). When strong
invariance holds and unique factor variances are
also equivalent across groups, then strict factorial
invariance is present (Meredith, 1993; Millsap &
Meredith, 2007). Strict factorial invariance implies
that systematic group differences in item means and
covariances are solely a function of group differ-
ences in factor means and covariances. According to
McArdle (2007) and Meredith and Horn (2001),
strong invariance should be expected to hold if two
subgroups are equivalent with respect to the latent
variable of interest; however, strict invariance is not
necessarily expected to hold.
Partial Invariance and Other
Complications
If measurement invariance is not met in studies
involving predictive relations among latent vari-
ables, then regression parameter bias will be present
(Humphreys, 1986; Millsap, 1998). If item inter-
cepts vary across groups, but the noninvariance is
ignored such that the noninvariant measurement
model is used to test structural hypotheses, then not
only will group differences in regression parameter
estimates in the structural part of the model repre-
sent any true group differences in the relations
between latent variables, but also the structural
parameter estimates will be confounded with group
differences in measurement. Thus, it is necessary to
 Item-Level Factor Analysis

test for measurement invariance before proceeding An Application of IFA

to tests of predictive models and making inferences
In this final section we provide a brief example of
about true group differences. Millsap (1998)
IFAs, involving less than continuously measured
described a formal test of measurement invariance
items, that merges the two IFA traditions that we
that is necessary to ensure that observed structural
have discussed. The data are from the Patient
differences are due to differences in the population
Reported Outcomes Measurement Information Sys-
rather than to differences in measurement. Millsap
tem (PROMIS), a multisite project that aims to
and Tein (2004) extended this work to include sim-
develop self-reported item banks for clinical
ple factor models with ordered categorical or dichot-
research. Although content domains in many areas
omous response types.
of health are included as part of this project, for the
Before providing an IFA application, it is impor-
purposes of our example, we focus on the emotional
tant to note that there are a variety of ways in which
distress domain.
item factor analytic methods can be conducted
Copyright American Psychological Association. Not for further distribution.

Anxiety and depressive symptoms items were

depending on the analyst’s goals. In situations in
split between two test administration forms. For
which researchers have little a priori knowledge of
brevity, we report only the results of Form 1, which
the factor structure of a set of items, the data ana-
had 759 children respond to 10 anxiety items and
lytic process might begin by fitting unrestricted EFA
10 depressive symptom items (Irwin et al., 2010).
models where items have as many loadings as fac-
All items had the same 5-point response scale with
tors (see Jöreskog, 1990; Jöreskog & Moustaki,
the options never (0), almost never (1), sometimes
2001; Muthén, 1984). It is more common in the IFA
(2), often (3), and almost always (4).
tradition to begin the data analysis process by fitting
Our first step was to examine the factor structure
models that are restrictions on the general EFA
of the individual items to determine whether there
framework (i.e., constraining some or many factor
was a single dimension underlying them (a precon-
loadings to be zero concurrent with the research
dition for unidimensional IRT). Initial EFA models
hypothesis). Often these models are variations on
were fit to the Form 1 data, as shown in Table 35.1.
hierarchical models, where all items receive a load-
The factor structure generally revealed depressive
ing from a general factor that is assumed to underlie
symptoms and anxiety factors but not simple struc-
all the items, with subsets of the items receiving
ture. A close inspection reveals many instances of
loadings that account for a shared association
items loading (in part) on the incorrect factor. Had
specific to the subset of items but that are above
commonly used techniques for deciding the number
and beyond the relationship accounted for by the
of factors been used (scree plot, fit indexes, magni-
general factor.
tude of factor loadings, inspection of eigenvalues),
As an alternative to bifactor or hierarchical mod-
two factors would have been extracted.
els, and useful in situations in which multidimension-
Taking an IFA approach to this problem would
ality may not be explicitly expected, there is often
consider item-level characteristics (e.g., item con-
utility in beginning the data analytic process by fitting
tent and the location of the item on the scale) in
a single factor model and then considering local
identifying subsets of locally dependent items while
dependence (LD) statistics as evidence of nuisance or
resolving dimensionality concerns. In such
extra dimensionality. This approach may be preferred
instances, a bifactor model, which estimates two
in situations in which the analyst does not begin with
loadings for all items, a nonzero general factor load-
the expectation of multidimensionality. In less com-
ing, and a group-specific loading (here, depressive
mon situations, where the analyst has no prior beliefs
symptoms and anxiety), serves as an excellent com-
about the structure of the items, EFA may still pro-
promise between an EFA and traditional simple-
vide a satisfactory starting position. In the application
structure CFA. In this case, a bifactor model is
that follows, we conduct EFA initially to show the
particularly relevant because the goal of the analysis
strengths and weakness of such an approach and then
is both to determine whether depressive symptoms
move into more traditional IFA models.

691
 Stucky, Gottfredson, and Panter
Table 35.1
Exploratory Factor Loadings for 20 PROMIS
Anxiety (A) and Depression (D) Items
Item
  A1. I got scared really easy. .88
  A2. I felt afraid. .89
  A3. I worried about what could happen to me. .65
  A4. It was hard for me to stop worrying. .59
  A5. I woke up at night scared. .55
  A6. I worried when I was away from home. .42
  A7. I was afraid that I would make mistakes. .34
Copyright American Psychological Association. Not for further distribution.
  A8. I felt nervous. .40
  A9. It was hard for me to relax. .30
A10. I felt afraid or scared. .64
  D1. I wanted to be by myself. −.08
  D2. I felt that no one loved me. .04
  D3. I cried more than usual. .39
  D4. I felt alone. .06
  D5. I felt like I couldn’t do anything right. .08
  D6. I felt so bad that I didn’t want to do .12
anything.
  D7. I felt everything in my life went wrong. .06
  D8. Being sad made it hard for me to do .28
things with my friends.
  D9. It was hard to do school work because I .25
felt sad.
D10. I felt like crying. .42
Note. Model fit using Crawford-Ferguson varimax
oblique rotation and mean- and variance-adjusted
weighted least squares estimation. The correlation
between factors is 0.55. χ2(88) = 529, comparative fit
index = 0.91, Tucker-Lewis index = 0.97, root-mean-
square error of approximation = 0.08.
and anxiety are best treated as separate dimensions,
and hence scales, while also identifying sources of
LD (i.e., violations of unidimensionality). Finally,
detecting LD in this manner allows the researcher to
control the dimensionality of the scale. If LD is
detected in either pairs or subsets of items, selecting
one item from each grouping, and setting aside the
LD-inducing items, should result in a unidimen-
sional set of items, as the residuals of the offending
items will no longer covary with the selected item.
If using FIML estimation, the Gibbons and Hedeker (1992) method would apply. In this context, integration would occur over eight orthogonal
5
dimensions (1 general, 2 domain specific, 1 subfactor, and 4 error correlations), which with four quadrature points per dimension amounts to 65,536
points. Cai (2010b) considered a similar analytic problem, and using a prototype of IRTPRO with the MH-RM algorithm, reached model convergence
in under 3 minutes compared with an ML/EM solution with adaptive quadrature, which took more than 4 hours.
692
A modified bifactor model was then fit using
Mplus with WLSMV estimation. This model is con-
sidered a modified version of the traditional bifactor
model because potential sources of LD identified in
the expanded EFA were modeled as subfactors (i.e.,
Factor three or more items receiving an additional factor
1 2 loading) and correlated errors, which represent cor-
−.12 relations between the residuals after accounting for
−.10 the covariance occurring for the general and domain-
.14 specific factor (Table 35.2).5 Goodness-of-fit indexes
.24 suggested that the augmented bifactor model fit the
.25
.27
data well: χ2(76, N = 621) = 247.83, comparative fit
.37 index = .95, Tucker-Lewis index = 0.99, root-mean-
.27 square error of approximation = .06.
.42 Used in this fashion, the bifactor model provides
.23
information regarding the intended dimensionality
.47
.76 and the presence of nuisance multidimensionality.
.44 The PROMIS researchers were initially interested in
.75 determining whether these data provided evidence
.75
that emotional distress was a single dimension, or
.64
whether distinguishable individual variation
.78 occurred between the anxiety items and then again
.58 between the depressive symptoms items. The fact
that substantial loadings differed significantly from
.59
zero on the group-specific factor for the depressive
.43 symptoms items in Table 35.2 indicated that the
covariation among the item responses could not be
adequately explained with the theory that a single
emotional distress dimension of individual differ-
ences underlies responses to all of the items. It is
a curiosity of the data that the general factor in
Table 35.2 is anxiety-dominated negative affect,
leaving little unique variance for the anxiety group-
specific factor. This fact is noted by comparing the
ratio of the general factor to the domain-specific fac-
tors and demonstrated in the large number of non-
significant loadings on the anxiety factor.
The analysis also highlights the precision with
which IFA can detect nuisance dimensionality.
Researchers who rely on EFA to determine the
dimensionality of the items may have settled on the
set of depressive symptoms and anxiety items origi-
nally hypothesized; however, a closer inspection
 Item-Level Factor Analysis

Table 35.2

Factor Loadings and Residual Correlations for an Augmented Bifactor Model Fitted to the Items on Form 1

Orthogonal group:
Specific factors
General Depressive Afraid/ Doublet residual
Item stem factor Anxiety symptoms scared correlations
I felt afraid. .68 .11 .67
I got scared really easy. .64 .30 .38
I felt afraid or scared. .76 .15 .20
It was hard for me to stop worrying. .73 .11 .32
I worried about what could happen to me. .70 .10
Copyright American Psychological Association. Not for further distribution.

I woke up at night scared. .72 .40

I was afraid that I would make mistakes. .70 -.39
It was hard for me to relax. .68 -.15
I worried when I was away from home. .64 .10
I felt nervous. .63 -.19
I felt everything in my life went wrong. .61 .56
I felt like I couldn’t do anything right. .62 .55
I felt so bad that I didn’t want to do anything. .57 .48
I felt alone. .59 .50 .27
I felt that no one loved me. .60 .47
Being sad made it hard for me to do things with my friends. .72 .28 .26
It was hard to do school work because I felt sad. .66 .31
I felt like crying. .66 .24 .49
I cried more than usual. .64 .23
I wanted to be by myself. .27 .30

Note. Italicized entries are less than 2 standard errors from 0. From “An Item Response Analysis of the Pediatric PROMIS
Anxiety and Depressive Symptoms Scales,” by D. E. Irwin, B. D. Stucky, M. M. Langer, D. Thissen, E. M. DeWitt, J. S.
Lai, J. W. Varni, K. Yeatts, and D. A. DeWalt, 2010, Quality of Life Research, 19, p. 600. Copyright 2010 by Springer
Science+Business Media. Reprinted with permission.

reveals that the general and group-specific factors
are not (all) conditionally independent. In Table
35.2, note that a cluster of items involves being
“scared or afraid” with responses that are more cor-
related than expected given the general factor and
the anxiety-specific factor, and four more pairs of
items have significant residual correlations. Begin-
ning at the top of Table 35.2, the pairs of items mod-
eled with residual correlations are about “worrying,”
“feelings of loneliness,” “sadness,” and “crying.”
Items in these pairs or triplets are (in part) like ask-
ing the same question twice. In each instance,
including a single item on the scale is sufficient and
providing both (or all three in the case of the triplet)
would violate assumptions of unidimensionality.
Conducting IFA in this careful manner is useful
for identifying and eliminating violations of local
independence. However, this modeling approach is
not without its own complexities. In the present
application, knowledge of the factor structure served
as a foundation for later unidimensional IRT param-
eter calibration (i.e., after setting aside locally
dependent items, the factors anxiety and depressive
symptoms were separately fit with unidimensional
IRT models). If the modified bifactor model is con-
sidered in a MIRT framework, however, many
difficult interpretation and scoring issues remain.
If IRT-based scores are desired, then, including
residual correlations, eight dimensions require
integration, and hence, eight possible scores. The
computation of scores for such models is underde-
veloped. Although MIRT scoring is slowly gaining
some use, uses typically do not involve cases with
small subsets of locally dependent items. With

693
 Stucky, Gottfredson, and Panter
current scoring techniques, the scores per factor
must be interpreted as conditional on the model’s
other latent variables. Interpretation of such scores
remains limited. When IFA is conducted to explore
or identify multidimensionality that often occurs in
psychological data, it serves as a useful alternative to
traditional EFA models. Had our previous analyses
concluded after the EFA, we would have missed a
great deal of LD. The situation shown in this data
example is not rare. In well-constructed, expert-
reviewed scales, LD is often missed, and it may be too
minor to be identified via an EFA but large enough to
affect item calibration. With recent advances in both
Copyright American Psychological Association. Not for further distribution.
efficient algorithms and computational speed, we
expect IFA to continue to grow as more researchers
become aware of the benefits of considering both
scale dimensionality and item effects.
References
Bandalos, D. L. (2008). Is parceling really necessary? A
comparison of results from item parceling and cat-
egorical variable methodology. Structural Equation
Modeling, 15, 211–240. doi:10.1080/10705510
801922340
Bartholomew, D. J. (1983). Latent variable models for
ordered categorical data. Journal of Econometrics, 22,
229–243. doi:10.1016/0304-4076(83)90101-X
Birnbaum, A. (1968). Some latent trait models and their
use in inferring an examinee’s ability. In F. M. Lord &
M. R. Novick (Eds.), Statistical theories of mental test
scores (pp. 395–479). Reading, MA: Addison-Wesley.
Bock, R. D. (1972). Estimating item parameters and
latent ability when responses are scored in two or
more nominal categories. Psychometrika, 37, 29–51.
doi:10.1007/BF02291411
Bock, R. D., & Aitkin, M. (1981). Marginal maximum
likelihood estimation of item parameters: An applica-
tion of the EM algorithm. Psychometrika, 46, 443–459.
doi:10.1007/BF02293801
Bock, R. D., & Lieberman, M. (1970). Fitting a
response model for n dichotomously scored items.
Psychometrika, 35, 179–197.
Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian
random effects model for testlets. Psychometrika, 64,
153–168. doi:10.1007/BF02294533
Cai, L. (2010a). High-dimensional exploratory item fac-
tor analysis by a Metropolis-Hastings Robbins-Monro
algorithm. Psychometrika, 75, 33–57.
Cai, L. (2010b). Metropolis-Hastings Robbins-Monro
algorithm for confirmatory item factor analysis.
694
Journal of Educational and Behavioral Statistics, 35,
307–335.
Cai, L., du Toit, S. H. C., & Thissen, D. (2011). IRTPRO:
Flexible, multidimensional, multiple
categorical IRT modeling [Computer software].
Chicago, IL: Scientific Software International.
Cattell, R. B., & Burdsal, C. A. (1975). The radial parcel
double factoring design: A solution to the item-vs.-
parcel controversy. Multivariate Behavioral Research,
10, 165–179. doi:10.1207/s15327906mbr1002_3
Chen, W. H., & Thissen, D. (1997). Local dependence indi-
ces for item pairs using item response theory. Journal
of Educational and Behavioral Statistics, 22, 265–289.
Childs, R. A., Dahlstrom, W. G., Kemp, S., & Panter, A. T.
(1992). Item response theory in personality assessment:
The MMPI-2 Depression Scale (Report 92-1). Chapel
Hill: Thurstone Psychometric Laboratory, University
of North Carolina at Chapel Hill.
Christofferson, A. (1975). Factor analysis of dichoto-
mized variables. Psychometrika, 40, 5–32.
doi:10.1007/BF02291477
Coffman, D. L., & MacCallum, R. C. (2005). Using par-
cels to convert path analysis models into latent vari-
able models. Multivariate Behavioral Research, 40,
235–259. doi:10.1207/s15327906mbr4002_4
DiStefano, C. (2002). The impact of categorization
with confirmatory factor analysis. Structural
Equation Modeling, 9, 327–346. doi:10.1207/
S15328007SEM0903_2
Edwards, M. C. (2010). A Markov chain Monte Carlo
approach to confirmatory item factor analysis.
Psychometrika, 75, 474–497.
Embretson, S. E., & Reise, S. P. (2000). Item response
theory for psychologists. Mahwah, NJ: Erlbaum.
Flora, D. B., & Curran, P. J. (2004). An empirical
evaluation of alternative methods of estimation
for confirmatory factor analysis with ordinal data.
Psychological Methods, 9, 466–491. doi:10.1037/
1082-989X.9.4.466
Gibbons, R. D., & Hedeker, D. R. (1992). Full-
information item bi-factor analysis. Psychometrika,
57, 423–436. doi:10.1007/BF02295430
Hagtvet, K. A., & Nasser, F. M. (2004). How well do
item parcels represent conceptually-defined latent
constructs? A two-facet approach. Structural
Equation Modeling, 11, 168–193. doi:10.1207/
s15328007sem1102_2
Hastings, W. K. (1970). Monte Carlo simulation meth-
ods using Markov chains and their applications.
Biometrika, 57, 97–109. doi:10.1093/biomet/57.1.97
Hau, K. T., & Marsh, H. W. (2004). The use of item par-
cels in structural equation modeling: Non-normal
data and small sample sizes. British Journal of
Mathematical and Statistical Psychology, 57, 327–351.
doi:10.1111/j.2044-8317.2004.tb00142.x

Heywood, H. B. (1931). On finite sequences of real num-
bers. Proceedings of the Royal Society: Series A, 134,
486–501. doi:10.1098/rspa.1931.0209
Horn, J. L., & McArdle, J. (1992). A practical and theo-
retical guide to measurement invariance in aging
research. Experimental Aging Research, 18, 117–144.
Hoskens, M., & De Boeck, P. (1997). A parametric model
for local dependence among test items. Psychological
Methods, 2, 261–277. doi:10.1037/1082-989X.2.3.261
Humphreys, L. G. (1986). An analysis and evaluation of
test and item bias in the prediction context. Journal of
Applied Psychology, 71, 327–333. doi:10.1037/0021-
9010.71.2.327
Irwin, D. E., Stucky, B. D., Langer, M. M., Thissen, D.,
Copyright American Psychological Association. Not for further distribution.
DeWitt, E. M., Lai, J. S., . . . DeWalt, D. A. (2010).
An item response analysis of the pediatric PROMIS
anxiety and depressive symptoms scales. Quality of
Life Research, 19, 595–607.
Jöreskog, K. G. (1969). A general approach to con-
firmatory maximum likelihood factor analy-
sis. Psychometrika, 34, 183–202. doi:10.1007/
BF02289343
Jöreskog, K. G. (1970). A general method for analysis of
covariance structures. Biometrika, 57, 239–251.
Jöreskog, K. G. (1990). New developments in LISREL:
Analysis of ordinal variables using polychoric cor-
relations and weighted least squares. Quality and
Quantity, 24, 387–404. doi:10.1007/BF00152012
Jöreskog, K. G., & Moustaki, I. (2001). Factor analysis of
ordinal variables: A comparison of three approaches.
Multivariate Behavioral Research, 36, 347–387.
doi:10.1207/S15327906347-387
Jöreskog, K. G., & Sörbom, D. (2001). LISREL user’s
guide. Chicago, IL: Scientific Software International.
Krosnick, J. A. (1999). Survey research. Annual Review
of Psychology, 50, 537–567. doi:10.1146/annurev.
psych.50.1.537
Lazarsfeld, P. F. (1950). The logical and mathematical
foundation of latent structure analysis. In S. A.
Stouffer (Ed.), Measurement and prediction.
Princeton, NJ: Princeton University Press.
Little, T. D., Cunningham, W. A., Shahar, G., &
Widaman, K. F. (2002). To parcel or not to par-
cel: Exploring the question, weighing the mer-
its. Structural Equation Modeling, 9, 151–173.
doi:10.1207/S15328007SEM0902_1
Little, T. D., Lindenberger, U., & Nesselroade, J. R.
(1999). On selecting indicators for multi-
variate measurement and modeling with latent
variables. Psychological Methods, 4, 192–211.
doi:10.1037/1082-989X.4.2.192
Lord, F. M. (1952). A theory of test scores. New York, NY:
Psychometric Society.
Item-Level Factor Analysis
Lord, F. M., & Novick, M. R. (1968). Statistical theories of
mental test scores. Reading, MA: Addison-Wesley.
McArdle, J. J. (2007). Five steps in the structural factor
analysis of longitudinal data. In R. Cudeck & R. C.
MacCallum (Eds.), Factor analysis at 100: Historical
developments and future directions (pp. 99–130).
Mahwah, NJ: Erlbaum.
McDonald, R. P. (1967). Nonlinear factor analysis.
Psychometric Monograph, J5.
McDonald, R. P. (1999). Test theory: A unified treatment.
Mahwah, NJ: Erlbaum.
McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty fac-
tors in binary data. British Journal of Mathematical
and Statistical Psychology, 27, 82–99.
Meredith, W. (1993). Measurement invariance, factor
analysis and factorial invariance. Psychometrika, 58,
525–543. doi:10.1007/BF02294825
Meredith, W., & Horn, J. L. (2001). The role of facto-
rial invariance in modeling growth and change. In
A. G. Sayer & L. M. Collins (Eds.), New methods for
the analysis of change (pp. 203–240). Washington,
DC: American Psychological Association.
doi:10.1037/10409-007
Millsap, R. E. (1997). Invariance in measurement
and prediction: Their relationship in the single-
factor case. Psychological Methods, 2, 248–260.
doi:10.1037/1082-989X.2.3.248
Millsap, R. E. (1998). Group differences in regression
intercepts: Implications for factorial invariance.
Multivariate Behavioral Research, 33, 403–424.
doi:10.1207/s15327906mbr3303_5
Millsap, R. E., & Kwok, O. (2004). Evaluating the impact
of partial factorial invariance on selection in two
populations. Psychological Methods, 9, 93–115.
doi:10.1037/1082-989X.9.1.93
Millsap, R. E., & Meredith, W. (2007). Factorial invari-
ance: Historical perspectives and new problems. In
R. Cudeck & R. C. MacCallum (Eds.), Factor analy-
sis at 100: Historical developments and future directions
(pp. 131–152). Mahwah, NJ: Erlbaum.
Millsap, R. E., & Tein, J. Y. (2004). Assessing facto-
rial invariance in ordered-categorical measures.
Multivariate Behavioral Research, 39, 479–515.
doi:10.1207/S15327906MBR3903_4
Mislevy, R. J. (1986). Recent developments in the fac-
tor analysis of categorical variables. Journal of
Educational Statistics, 11, 3–31. doi:10.2307/1164846
Muraki, E. (1992). A generalized partial credit
model: Application of an EM algorithm. Applied
Psychological Measurement, 16, 159–176.
doi:10.1177/014662169201600206
Muthén, B. O. (1978). Contributions to factor analysis of
dichotomous variables. Psychometrika, 43, 551–560.
doi:10.1007/BF02293813
695
 Stucky, Gottfredson, and Panter
Muthén, B. (1983). Latent variable structural equation mod-
eling with categorical data. Journal of Econometrics, 22,
43–65. doi:10.1016/0304-4076(83)90093-3
Muthén, B. O. (1984). A general structural equation
model with dichotomous, ordered categorical, and
continuous latent variable indicators. Psychometrika,
49, 115–132. doi:10.1007/BF02294210
Muthén, B. O., & Asparouhov, T. (2002). Latent variable
analysis with categorical outcomes: Multiple-group
and growth modeling in Mplus. Mplus Web Note
No. 4. Retrieved from http://www.statmodel.com/
examples/webnote.shtml
Muthén, B. O., du Toit, S. H. C., & Spisic, D. (1997).
Robust inference using weighted least squares and
quadratic estimating equations in latent variable
Copyright American Psychological Association. Not for further distribution.
modeling with categorical and continuous outcomes.
Unpublished manuscript. Retrieved from http://www.
gseis.ucla.edu/faculty/muthen/psychometrics.htm
Muthén, B. O., & Lehman, J. (1985). Multiple group
IRT modeling: Application to item bias analy-
sis. Journal of Educational Statistics, 10, 133–142.
doi:10.2307/1164840
Muthén, L. K., & Muthén, B. O. (1998–2007). Mplus
user’s guide (5th ed.). Los Angeles, CA: Muthén &
Muthén.
Panter, A. T., Swygert, K., Dahlstrom, W. G., & Tanaka,
J. S. (1997). Factor analytic models for item-level
personality data. Journal of Personality Assessment,
68, 561–589. doi:10.1207/s15327752jpa6803_6
Panter, A. T., & Tanaka, J. S. (1987, April). Statistically
appropriate methods for analyzing dichotomous data:
Assessing self-monitoring. Paper presented at the
Eastern Psychological Association, Arlington, VA.
Panter, A. T., Tanaka, J. S., & Wellens, T. R. (1992).
The psychometrics of order effects. In S. Sudman &
N. Schwarz (Eds.), Context effects in social and psy-
chological research (pp. 249–264). New York, NY:
Springer-Verlag.
Rasch, G. (1960). Probabilistic models for some intelli-
gence and attainment tests. Chicago, IL: University of
Chicago Press.
Reckase, M. D. (1979). Unifactor latent trait models
applied to multifactor tests: Results and implica-
tions. Journal of Educational Statistics, 4, 207–230.
doi:10.2307/1164671
Reckase, M. D. (2009). Multidimensional item response
theory. New York, NY: Springer.
Reise, S. P., & Waller, N. G. (1990). Fitting the two-
parameter model to personality data. Applied
Psychological Measurement, 14, 45–58. doi:10.1177/
014662169001400105
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993).
Confirmatory factor analysis and item response
theory: Two approaches for exploring measurement
696
invariance. Psychological Bulletin, 114, 552–566.
doi:10.1037/0033-2909.114.3.552
Robbins, H., & Monro, S. (1951). A stochastic approxi-
mation method. Annals of Mathematical Statistics, 22,
400–407. doi:10.1214/aoms/1177729586
Samejima, F. (1969). Estimation of latent ability using
a response pattern of graded scores, Psychometrika,
Monograph No. 17.
Satorra, A., & Bentler, P. M. (1994). Corrections to test
statistic and standard errors in covariance struc-
ture analysis. In A. von Eye & C. C. Clogg (Eds.),
Analysis of latent variables in developmental research
(pp. 399–419). Newbury Park, CA: Sage.
Schaeffer, N. C. (1988). An application of the item
response theory to the measurement of depression.
In C. C. Clogg (Ed.), Sociological methodology
(pp. 271–307). Washington, DC: American
Sociological Association.
Schilling, S., & Bock, R. D. (2005). High-dimensional
maximum marginal likelihood item factor analysis by
adaptive quadrature. Psychometrika, 70, 533–555.
Steenkamp, J. E. M., & Baumgartner, H. (1998).
Assessing measurement invariance in cross-national
consumer research. Journal of Consumer Research, 25,
78–107. doi:10.1086/209528
Steinberg, L. (1994). Context and serial-order effects in
personality measurement: Limits on the general-
ity of measuring changes the measure. Journal of
Personality and Social Psychology, 66, 341–349.
doi:10.1037/0022-3514.66.2.341
Steinberg, L., & Thissen, D. (1995). Item response theory
in personality research. In P. E. Shrout & S. Fiske
(Eds.), Personality research, methods, and theory: A
Festschrift honoring Donald W. Fiske (pp. 161–181).
Hillsdale, NJ: Erlbaum.
Stout, W. F. (1987). A nonparametric approach for
assessing latent trait dimensionality. Psychometrika,
52, 589–617. doi:10.1007/BF02294821
Takane, Y., & de Leeuw, J. (1987). On the relationship
between item response theory and factor analysis of
discretized variables. Psychometrika, 52, 393–408.
doi:10.1007/BF02294363
Tanaka, J. S., Panter, A. T., & Winborne, W. C. (1988).
Dimensions of the need for cognition: Subscales and
gender differences. Multivariate Behavioral Research,
23, 35–50. doi:10.1207/s15327906mbr2301_2
Thissen, D. (1992, August). Item response theory in psy-
chological research. Invited address at the 100th
Annual Convention of the American Psychological
Association, Washington, DC.
Thissen, D. (2003). MULTILOG 7 user’s guide. Chicago,
IL: Scientific Software International.
Thissen, D., & Steinberg, L. (1986). A taxonomy of
item response models. Psychometrika, 51, 567–577.
doi:10.1007/BF02295596

Thissen, D., Steinberg, L., & Mooney, J. (1989). Trace
lines for testlets: A use of multiple-categorical
response models. Journal of Educational Measurement,
26, 247–260. doi:10.1111/j.1745-3984.1989.
tb00331.x
Thurstone, L. L. (1947). Multiple factor analysis. Chicago,
IL: University of Chicago Press.
Wainer, H., Bradlow, E. T., & Wang, X. (2007). Testlet
response theory and its applications. New York, NY:
Cambridge University Press.
Wainer, H., & Kiely, G. L. (1987). Item clusters and
computerized adaptive testing: A case for testlets.
Journal of Educational Measurement, 24, 185–201.
doi:10.1111/j.1745-3984.1987.tb00274.x
Copyright American Psychological Association. Not for further distribution.
Waller, N. G., & Reise, S. P. (1989). Computerized adap-
tive personality assessment: An illustration with the
absorption scale. Journal of Personality and Social
Psychology, 57, 1051–1058. doi:10.1037/0022-3514.
57.6.1051
West, S. G., Finch, J. F., & Curran, P. J. (1995).
Structural equation models with non-normal vari-
ables: Problems and remedies. In R. Hoyle (Ed.),
Item-Level Factor Analysis
Structural equation modeling: Concepts, issues, and
applications (pp. 56–75). Newbury Park, CA: Sage.
Widaman, K. F., & Reise, S. P. (1997). Exploring the mea-
surement invariance of psychological instruments:
Applications in the substance use domain. In K. J.
Bryant, M. Windle, & S. G. West (Eds.), The science of
prevention: Methodological advances from alcohol and
substance abuse research (pp. 281–324). Washington,
DC: American Psychological Association. doi:10.1037/
10222-009
Wirth, R. J., & Edwards, M. C. (2007). Item factor
analysis: Current approaches and future directions.
Psychological Methods, 12, 58–79. doi:10.1037/1082-
989X.12.1.58
Yen, W. M. (1984). Effects of local item dependence
on the fit and equating performance of the three-
parameter logistic model. Applied Psychological
Measurement, 8, 125–145. doi:10.1177/0146621
68400800201
Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D.
(2003). BILOG-MG3 user’s guide. Chicago, IL:
Scientific Software International.
697