FSU 2016SU Xia Fsu 0071E 13379

Florida State University Libraries
2016
Investigating the Chi-Square-Based Model-

Fit Indexes for WLSMV and ULSMV
Estimators
Yan Xia
Follow this and additional works at the FSU Digital Library. For more information, please contact lib-ir@fsu.edu
FLORIDA STATE UNIVERSITY
COLLEGE OF EDUCATION
INVESTIGATING THE CHI-SQUARE-BASED MODEL-FIT INDEXES FOR
WLSMV AND ULSMV ESTIMATORS
By
YAN XIA
A Dissertation submitted to the

Department of Educational Psychology and Learning Systems
in partial fulfillment of the
requirements for the degree of
Doctor of Philosophy
2016
Yan Xia defended this dissertation on July 5, 2016.
The members of the supervisory committee were:
Yanyun Yang
Professor Directing Dissertation
Fred W. Huffer
University Representative
Russell G. Almond
Committee Member
Betsy J. Becker
Committee Member
Insu Paek
Committee Member
The Graduate School has verified and approved the above-named committee members, and
certifies that the dissertation has been approved in accordance with university requirements.
ii
This work is dedicated to Guoming Yang
iii
ACKNOWLEDGMENTS
Words are powerless to express my deepest gratitude to my communities of scholars,

friends, and family.
I would like to thank my academic chair Dr. Yanyun Yang from the bottom of my heart.
She has been acting as a positive role model by generously sharing her enthusiasm, expertise,
and skills in this field with me. She is invested in the success of my career, helps me find
gratification in my chosen profession, and empowers me to develop my research skills and
passion. I truly enjoy the collaborative research with her. I am also grateful to my committee
members, Dr. Russell G. Almond, Dr. Betsy J. Becker, and Dr. Insu Paek, for their dedication to
help me improve the quality of my dissertation. I would also like to express my appreciation to
my outside committee member Dr. Fred W. Huffer, from who I’ve acquired critical knowledge
of statistical theories and sincere support on my dissertation. In addition, I feel grateful to Dr.
Xufeng Niu for giving me valuable suggestions on my career and being my advisor when I dual-
enrolled as a Master student in the Department of Statistics, Florida State University.
I would not have finished my doctoral study without those who have provided me with
financial support and priceless study, research, and work opportunities. I wish to express my
sincere appreciation to Dr. Yiu-Fai Yung for his mentorship when I was a summer intern at SAS
Institute Inc., Dr. Faranak Rohani for the research assistant position at the Center for
Advancement of Learning and Assessment, and Dr. Fabian Tata for the tutorial assistant position
at the Center for Academic Retention and Enhancement.
Last, I wish to thank my family and dear friends for their love and endless spiritual
support. My sincere appreciation goes to Guoming, Zili, and Ping.
iv
TABLE OF CONTENTS
List of Tables ................................................................................................................................ vii

List of Figures ................................................................................................................................ ix
Abstract ............................................................................................................................................x
INTRODUCTION ...........................................................................................................................1
WLSMV and ULSMV Estimators ..............................................................................................2

Population-corrected and Sample-corrected Robust Model Fit Indexes.....................................4
Research Questions .....................................................................................................................7
REVEW OF LITERATURE..........................................................................................................10
Confirmatory Factor Analysis with Ordered Categorical Variables .........................................10

Robust Model-fit Indexes for WLSMV and ULSMV ..............................................................16
Previous Studies on the Robust Model-fit Indexes for WLSMV and ULSMV........................20
Purposes of Study......................................................................................................................24
METHODS ....................................................................................................................................26
Simulation at the Population Level ...........................................................................................26

Simulation at the Sample Level ................................................................................................29
RESULTS ......................................................................................................................................31
Results at the Population Level .................................................................................................31

Results at the Sample Level ......................................................................................................33
DISCUSSION AND CONCLUSIONS .........................................................................................49
The Dependency of Model Fit Indexes on Thresholds .............................................................50

The Accuracy of Model Fit Indexes..........................................................................................51
The Use of Conventional Cutoff Values ...................................................................................52
Future Research.........................................................................................................................53
Conclusions ...............................................................................................................................53
APPENDICES ...............................................................................................................................55
A. RESULTS FROM THE SIMULATION AT THE POPULATION LEVEL............................55

B. RESULTS FROM THE SIMULATION AT THE SAMPLE LEVEL .....................................67
C. INADMISSIBLE SOLUTION RATES AT THE SAMPLE LEVEL ......................................86
References ......................................................................................................................................90
v
Biographical Sketch .......................................................................................................................95
vi
LIST OF TABLES
1 Means (Standard Deviations) of RMSEA for Model 1, 2-point Scale, and 9 Items................43
3 Means (Standard Deviations) of CFI for Model 1, 2-point Scale, and 9 Items .......................45
5 Means (Standard Deviations) of TLI for Model 1, 2-point Scale, and 9 Items. ......................47
6 Means (Standard Deviations) of TLI for Model 1, 4-point Scale, and 9 Items .......................48
7 Means (Standard Deviations) of RMSEA for Model 1, 2-point Scale, and 18 Items..............68
13 Means (Standard Deviations) of CFI for Model 1, 2-point Scale, and 18 Items .....................74
17 Means (Standard Deviations) of CFI for Model 2, 2-point Scale, and 18 Items. ....................78
19 Means (Standard Deviations) of TLI for Model 1, 2-point Scale, and 18 Items .....................80
20 Means (Standard Deviations) of TLI for Model 1, 4-point Scale, and 18 Items. ....................81
vii
23 Means (Standard Deviations) of TLI for Model 2, 2-point Scale, and 18 Items. ....................84
24 Means (Standard Deviations) of TLI for Model 2, 4-point Scale, and 18 Items .....................85
25 Inadmissible Solution Rates (%) for Model 1 with 9 Items.....................................................86
26 Inadmissible Solution Rates (%) for Model 1 with 18 Items...................................................87
27 Inadmissible Solution Rates (%) for Model 2 with 9 Items.....................................................88
28 Inadmissible Solution Rates (%) for Model 2 with 18 Items...................................................89
viii
LIST OF FIGURES
1 RMSEA for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories ................37
2 RMSEA for Model 1, Sample Size 1,000,000, 9 Items, and 4 Ordered Categories ................38
3 CFI for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories .......................39
4 CFI for Model 1, Sample Size 1,000,000, 9 Items, and 4 Ordered Categories .......................40
5 TLI for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories .......................41
6 TLI for Model 1, Sample Size 1,000,000, 9 Items, and 4 Ordered Categories .......................42
7 RMSEA for Model 1, Sample Size 1,000,000, 18 Items, and 2 Ordered Categories ..............55
8 RMSEA for Model 1, Sample Size 1,000,000, 18 Items, and 4 Ordered Categories ..............56
9 RMSEA for Model 2, Sample Size 1,000,000, and 2 Ordered Categories ..............................57
10 RMSEA for Model 2, Sample Size 1,000,000, and 4 Ordered Categories ..............................58
11 CFI for Model 1, Sample Size 1,000,000, 18 Items, and 2 Ordered Categories .....................59
12 CFI for Model 1, Sample Size 1,000,000, 18 Items, and 4 Ordered Categories .....................60
13 CFI for Model 2, Sample Size 1,000,000, and 2 Ordered Categories .....................................61
14 CFI for Model 2, Sample Size 1,000,000, and 4 Ordered Categories .....................................62
15 TLI for Model 1, Sample Size 1,000,000, 18 Items, and 2 Ordered Categories .....................63
16 TLI for Model 1, Sample Size 1,000,000, 18 Items, and 4 Ordered Categories .....................64
17 TLI for Model 2, Sample Size 1,000,000, and 2 Ordered Categories .....................................65
18 TLI for Model 2, Sample Size 1,000,000, and 4 Ordered Categories .....................................66
ix
ABSTRACT
In structural equation modeling (SEM), researchers use the model chi-square statistic and
model-fit indexes to evaluate model-data fit. Root mean square error of approximation
(RMSEA), comparative fit index (CFI), and Tucker-Lewis index (TLI) are widely applied
model-fit indexes.
When data are ordered and categorical, the most popular estimator is the diagonally
weighted least squares (DWLS) estimator. Robust corrections have been proposed to adjust the
uncorrected chi-square statistic from DWLS so that its first and second order moments are in
alignment with the target central chi-square distribution under correctly specified models. DWLS
with such a correction is called the mean- and variance-adjusted weighted least squares
(WLSMV) estimator. An alternative to WLSMV is the mean-and variance-adjusted unweighted
least squares (ULSMV) estimator, which has been shown to perform as well as, or slightly better
than WLSMV.
Because the chi-square statistic is corrected, the chi-square-based RMSEA, CFI, and TLI
are thus also corrected by replacing the uncorrected chi-square statistic with the robust chi-
square statistic. The robust model fit indexes calculated in such a way are named as the
population-corrected robust (PR) model fit indexes following Brosseau-Liard, Savalei, and Li
(2012). The PR model fit indexes are currently reported in almost every application when
WLSMV or ULSMV is used. Nevertheless, previous studies have found the PR model fit
indexes from WLSMV are sensitive to several factors such as sample sizes, model sizes, and
thresholds for categorization.
The first focus of this dissertation is on the dependency of model fit indexes on the
thresholds for ordered categorical data. Because the weight matrix in the WLSMV fit function
and the correction factors for both WLSMV and ULSMV include the asymptotic variances of
thresholds and polychoric correlations, the model fit indexes are very likely to depend on the
thresholds. The dependency of model fit indexes on the thresholds is not a desirable property,
because when the misspecification lies in the factor structures (e.g., cross loadings are ignored or
two factors are considered as a single factor), model fit indexes should reflect such
misspecification rather than the threshold values.
x
As alternatives to the PR model fit indexes, Brosseau-Liard et al. (2012), Brosseau-Liard
and Savalei (2014), and Li and Bentler (2006) proposed the sample-corrected robust (SR) model
fit indexes. The PR fit indexes are found to converge to distorted asymptotic values, but the SR
fit indexes converge to their definitions asymptotically. However, the SR model fit indexes were
proposed for continuous data, and have been neither investigated nor implemented in SEM
software when WLSMV and ULSMV are applied.
This dissertation thus investigates the PR and SR model fit indexes for WLSMV and
ULSMV. The first part of the simulation study examines the dependency of the model fit indexes
on the thresholds when the model misspecification results from omitting cross-loadings or
collapsing factors in confirmatory factor analysis. The study is conducted on extremely large
computer-generated datasets in order to approximate the asymptotic values of model fit indexes.
The results find that only the SR fit indexes from ULSMV are independent of the population
threshold values, given the other design factors. The PR fit indexes from ULSMV, and the PR
and SR fit indexes from WLSMV are influenced by thresholds, especially when data are binary
and the hypothesized model is greatly misspecified.
The second part of the simulation varies the sample sizes from 100 to 1000 to investigate
whether the SR fit indexes under finite samples are more accurate estimates of the defined values
of RMSEA, CFI, and TLI, compared with the uncorrected model fit indexes without robust
correction and the PR fit indexes. Results show that the SR fit indexes are the more accurate in
general. However, when the thresholds are different across items, data are binary, and sample
size is less than 500, all versions of these indexes can be very inaccurate. In such situations,
larger sample sizes are needed.
In addition, the conventional cutoffs developed from continuous data with maximum
likelihood (e.g., RMSEA < .06, CFI > .95, and TLI > .95; Hu & Bentler, 1999) have been
applied to WLSMV and ULSMV regardless of the arguments against such a practice (e.g.,
Marsh, Hau, & Wen, 2004). For comparison purposes, this dissertation reports the RMSEA, CFI,
and TLI based on continuous data using maximum likelihood before the variables are
categorized to create ordered categorical data. Results show that the model fit indexes from
maximum likelihood are very different from those from WLSMV and ULSMV, suggesting that
the conventional rules should not be applied to WLSMV and ULSMV.
xi
CHAPTER 1
INTRODUCTION
In educational and behavioral research, many variables are latent and are not directly
observable. Structural equation modeling (SEM) is a family of statistical techniques that
investigate the relationships among observed and latent variables. Researchers start with a
hypothesized model representing the inter-relationships among variables, and then evaluate if the
hypothesized model fits the data well. Evaluation of model-data fit in SEM thus becomes a
critical issue. A model is said to fit the data if the relationships specified in the model adequately
reproduce the relationships existing in the data.
It has been a conventional practice to evaluate model-data fit using the chi-square statistic
along with several model-fit indexes. The chi-square statistic is calculated as (n - 1)∙F, where n is
the sample size and F is a fit function characterizing the discrepancy between the observed
covariance matrix and the model-implied covariance matrix. The chi-square statistic is used to
test the null hypothesis that the unstructured covariance matrix equals to the model-implied
covariance matrix in the population. When data are continuous and multivariate normally
distributed, and when the model is correctly specified, the chi-square statistic obtained from the
normal-theory estimators such as the maximum likelihood (ML) estimator has a central chi-
square distribution with its expectation equal to the model degrees of freedom (e.g., Bollen,
1989).
In reality, hypothesized models are frequently misspecified. Instead, researchers often
search for a model that explains a sufficient amount of inter-relationships among the variables. In
other words, an “acceptable misspecified model” is often chosen. Researchers are more
interested in knowing how likely it is that a model fit is worse than an acceptable fit. When the
model is misspecified, the chi-square statistic has a non-central chi-square distribution, and it
inflates substantially when the sample size increases (e.g., Curran, West, & Finch, 1996), leading
to different model-fit decisions given the same degree of model misspecification. The chi-square
statistic is therefore a less desirable statistic than it is supposed to be in practice.
In addition to the chi-square statistic, many model-fit indexes are available in the SEM
literature (Bentler & Bonett, 1980). These indexes provide supplemental descriptions regarding
1
model-data fit. The present study considers three commonly applied model-fit indexes, all of
which are functions of the model chi-square statistic but which are less subject to the inflation
due to large sample sizes under misspecified models. They are root mean square error of
approximation (RMSEA; Cudeck & Browne, 1992; Steiger, 1990; Steiger & Lind, 1980),
comparative fit index (CFI; Bentler, 1990), and Tucker-Lewis index (TLI; Bentler & Bonett,
1980; Tucker & Lewis, 1973). RMSEA quantifies how far the hypothesized model is from
perfect fit to the data. Therefore, RMSEA is referred to as an absolute measure of model-data fit.
CFI and TLI compare the fit of the hypothesized model with that of the baseline model. The
baseline model assumes zero covariance among the variables (note that different SEM software
programs define the baseline model in different ways; in Mplus (Muthén & Muthén, 1998-2015),
the covariances among exogenous variables are estimated in the baseline model). Because the
baseline model yields the worst model-data fit, CFI and TLI are referred to as incremental fit
indexes. RMSEA, CFI, and TLI are all defined as functions of the fit function in the population,
and thus also of the model chi-square statistic.
Ideally, when the hypothesized model is correctly specified, the population values of
RMSEA, CFI, and TLI equal to 0, 1, and 1, respectively. In practice, researchers frequently use a
set of cutoff values to make decisions regarding model-data fit. For example, when RMSEA <
.06, CFI > .95, and TLI > .95 (Hu & Bentler, 1999), a hypothesized model is considered as
acceptable. It is important that the values of these fit indexes reflect the degree to which a
hypothesized model is misspecified, and the cutoff values yield sufficient power to reject a
model which does not adequately explain the relationships among the variables. Furthermore,
because the definitions of RMSEA, CFI, and TLI are functions of the fit function asymptotically,
it is important that as the sample size increases to infinity, RMSEA, CFI, and TLI converge to
their defined values.
WLSMV and ULSMV Estimators
In practice, researchers frequently collect data which are ordered and categorical,
violating the normality assumption underlying the normal-theory estimators. For example,
survey research often uses items anchored on Likert scales with 2 to 11 possible response
categories. Previous studies (e.g., Beauducel & Herzberg, 2006; Koziol, 2010; Rhemtulla,
Brosseau-Liard, & Savalei, 2012) showed that applying the normal-theory estimators to ordered
2
categorical data results in biased parameter estimates and inaccurate standard errors, especially
when the number of ordered categories is less than four.
The diagonally weighted least squares (DWLS) estimator based on polychoric
correlations and thresholds is more appropriate for analyzing ordered categorical data in SEM.
One of the key assumptions in DWLS is that an unobserved continuous variable is underlying
the ordered categorical variable. The polychoric correlation is used as the measure of the
relationship between two underlying continuous variables, and is estimated from the observed
ordered categorical data. When data are binary, the correlation between the two underlying
continuous variables is termed tetrachoric correlation, which is a special case of the polychoric
correlation. The DWLS estimator uses a diagonal weight matrix in parameter estimation. It is
much less computationally demanding than the weighted least squares (WLS) estimator which
uses the full weight matrix (Muthén, du Toit, & Spisic, 1997). The full weight matrix in WLS
contains the asymptotic variances and covariances of the thresholds and polychoric correlations.
The diagonal weight matrix in DWLS utilizes only the asymptotic variances of the thresholds
and polychoric correlations. Simulation studies showed that the DWLS estimator is superior to
ML and WLS when data are ordered and categorical, regarding parameter and standard error
estimation accuracy (Flora & Curran, 2004; Muthén et al., 1997; Oranje, 2003; Yang-Wallentin,
Jöreskog, & Luo, 2010).
However, because the DWLS estimator employs a diagonal weight matrix, its chi-square
statistic deviates from the central target central chi-square distribution (i.e., the chi-square
distribution with the expectation equal to the model degrees of freedom) under correctly
specified models. Robust methods have been proposed to adjust the chi-square statistic to be
more consistent with the target chi-square distribution. One of the robust methods is called the
mean- and variance-adjusted weighted least squares (WLSMV; Asparouhov & Muthén, 2010;
Muthén, 1984; Muthén et al., 1997). WLSMV adjusts the uncorrected chi-square statistic so that
the corrected chi-square has its mean and variance equal to those of the target central chi-square
distribution asymptotically when the model is correctly specified. WLSMV is implemented in
Mplus and has become the most common method in the SEM literature to analyze ordered
categorical variables (Savalei & Rhemtulla, 2013).
As an alternative to the DWLS estimator, the unweighted least squares (ULS) estimator
has also been recommended to analyze ordered categorical data in SEM (Forero et al., 2009), but
3
it has received less attention than the DWLS estimator. Similar to DWLS, ULS provides model
parameter estimations based on the polychoric correlation matrix and thresholds. However, the
weight matrix in the ULS fit function is further simplified to be an identity matrix. The mean-
and variance-adjusted ULS (ULSMV) is the robust ULS estimator implemented in Mplus, where
the chi-square statistic from ULS is corrected so that its mean and variance equal to those of the
target central chi-square distribution when the hypothesized model is correctly specified.
Although ULSMV has been less applied in substantive research, previous studies found that it
provides parameter and standard error estimations as accurate as, or slightly better than, those
from WLSMV under many conditions (Forero et al., 2009; Li, 2014; Yang-Wallentin et al.,
2010). Savalei and Rhemtulla (2013) found that the robust chi-square statistic from ULSMV
outperformed that from WLSMV regarding Type I error rates and power. The major
disadvantage is that ULSMV encounters a higher model nonconvergence rate than WLSMV
(Forero et al., 2009).
Population-corrected and Sample-corrected Robust Model Fit Indexes
Because RMSEA, CFI, and TLI are all functions of the model chi-square statistic, when
the uncorrected chi-square statistic from DWLS or ULS is adjusted, it leads to the natural choice
that RMSEA, CFI, and TLI should also be corrected. Currently, SEM software programs provide
the robust RMSEA, CFI, and TLI by simply replacing the uncorrected chi-square statistic with
the WLSMV or ULSMV robust chi-square in the formulas of RMSEA, CFI, and TLI. I call the
resulting robust model-fit indexes the population-corrected robust (PR) model-fit indexes and
label them as RMSEAPR, CFIPR, and TLIPR in this dissertation. The name “PR model-fit indexes”
was initially provided by Brosseau-Liard, Savalei, and Li (2012) in the context of ML with
Satorra and Bentler’s (1994) correction for continuous data, because the PR fit indexes change
the population values of RMSEA, CFI, and TLI.
For the ML estimator, researchers have been using the conventional cutoffs (e.g.,
RMSEA < .06, CFI > .95, and TLI > .95; Hu & Bentler, 1999) to determine an acceptable
model-data fit. Such cutoffs were established originally for continuous data, where ML was
applied (Hu & Bentler, 1999). However, Hu and Bentler (1999) did not recommend the
application of the conventional cutoffs universally, and they suggested to use a two-index
strategy. However, practitioners are tempted to apply Hu and Bentler’s cutoffs to a variety of
4
models, for continuous and categorical data, and with different estimation methods. Several
simulation studies (Garrido, Abad, & Ponsoda, 2015; Koziol, 2010; Li, 2014; Yang & Xia, 2015;
Yu, 2002) have evaluated whether the conventional rules can be similarly applied to WLSMV.
Findings from these studies suggest that the RMSEAPR, CFIPR, and TLIPR in WLSMV perform
very differently from those in ML. Although it seems appealing in the first place to establish a
new set of cutoff values for WLSMV, previous studies (e.g., Garrido et al., 2015; Yang & Xia,
2015; Yu, 2002) found that it is rather difficult to do so, because the behaviors of the PR model-
fit indexes are unclear, and vary by conditions such as sample sizes, model sizes, and thresholds
for categorization.
One focus of this dissertation is on the dependency of model fit indexes on the thresholds
when ordered categorical data are analyzed. Such a dependency is not a desirable property for
model-data fit evaluation, because model fit indexes should only measure the model-data
discrepancy rather than be influenced by the threshold values. For example, when the population
model has two distinct latent factors that are correlated, a researcher may analyze the data using
a single factor model. The misspecification in this example is due to the misspecification of the
number of factors, and has nothing to do with the thresholds. Therefore, the resulting model-data
fit should reflect how much misfit is from the under-specification of factor dimensions, and the
values of the model fit indexes shouldn’t change when the population threshold values change.
In addition, if the model fit indexes depend on the thresholds, their corresponding cutoff values
should also be threshold specific, which is impractical.
One possible explanation for the dependency of the PR model-fit indexes from WLSMV
on the thresholds is that the DWLS fit function has a diagonal weight matrix containing the
asymptotic variances of the thresholds and polychoric correlations, both of which are influenced
by the population threshold values (Jöreskog, 1994; Olsson, 1979). Therefore, the PR model-fit
indexes from WLSMV can be confounded with the thresholds when the model is misspecified,
and may not be solely measures of the model-data discrepancy. Previous studies (e.g., Beauducel
& Herzberg, 2006; Garrido et al., 2015; Yang & Xia, 2015; Yu, 2002) investigated the PR fit
indexes under small to moderate sample sizes. It is unknown if their dependency on the
thresholds is due to the property of the DWLS fit function or to sample fluctuation.
A second possible explanation lies in the fact that the RMSEAPR, CFIPR, and TLIPR from
WLSMV do not converge to the population definitions of RMSEA, CFI, and TLI as the sample
5
size increases to infinity. Brosseau-Liard et al. (2012) and Brosseau-Liard and Savalei (2014)
pointed out that, for continuous data analyzed using ML with Satorra and Bentler’s (1994) robust
correction, the PR model-fit indexes converge to different values from the population definitions
of RMSEA, CFI, and TLI when models are misspecified. One may expect that the PR model-fit
indexes from WLSMV also have such a problem, because they adjust the chi-square statistic
using similar logic to that behind Satorra and Bentler’s (1994) robust correction (Savalei, 2014).
The asymptotic values of the PR model-fit indexes contain the scale and shift factors from the
chi-square robust correction (Asparouhov & Muthén, 2010). Because these factors also involve
the asymptotic variances of the thresholds and polychoric correlations, the PR model-fit indexes
are likely to depend on the threshold values when the model is misspecified.
As an example, I generated a binary data set with 1,000,000 observations under a 3-factor
CFA model with the factor correlations equal to .3, and the data were fitted to a 1-factor model
using WLSMV. More details of the data generation process can be found in the description of
model 1 shown in Chapter 3. The value of RMSEAPR for this large sample changed from .124 to
.092 and then to .034 when the thresholds increased from 0 to 1, and then to 2. In this example,
given the same degree of model misspecification, as the thresholds increase, this misspecified
model is more likely to be considered as acceptable. This is problematic because ideally a model-
fit index should reflect only model-data misfit, rather than any other factor. However, the
RMSEAPR calculated based on the DWLS fit function is confounded with the thresholds for
categorization. As I demonstrate in Chapter 2, CFIPR and TLIPR are also subject to the thresholds
for categorization.
Alternative to the PR fit indexes, Li and Bentler (2006), Brosseau-Liard et al. (2012), and
Brosseau-Liard and Savalei (2014) proposed the sample-corrected robust (SR) model-fit indexes
for the Satorra and Bentler’s (1994) robust chi-square applied for nonnormal continuous data
with the normal-theory ML estimator. Brosseau-Liard et al. (2012) and Brosseau-Liard and
Savalei (2014) showed that the major advantage of the SR model-fit indexes over their PR
counterparts is that the SR fit indexes converge to the population definitions of RMSEA, CFI,
and TLI when the model is misspecified. Therefore, the name of the SR model-fit indexes come
from the fact that they correct the model fit indexes at the sample level, but they don’t correct
their population values. No study so far has extended the SR model-fit indexes to the estimators
for ordered categorical data. As shown in Chapter 2, the asymptotic values of the SR fit indexes
6
for WLSMV do not contain the scale and shift factors. Nevertheless, because the fit function of
DWLS itself depends on the thresholds, the SR fit indexes for WLSMV are still expected to be
influenced by the threshold values.
For ULSMV, no research has systematically evaluated its model-fit indexes as far as I am
aware. The weight matrix in ULS is an identity matrix, and thus the thresholds are irrelevant to
its fit function asymptotically. However, the scale and shift factors in the chi-square robust
correction contain the asymptotic variances of the thresholds and polychoric correlations. The
ULSMV PR model-fit indexes are therefore expected to depend on the thresholds. Because the
asymptotic values of the ULSMV SR model-fit indexes do not contain the scale and shift factors,
they are not expected to be confounded with the thresholds.
Research Questions
A review of previous studies (e.g., Beauducel & Herzberg, 2006; DiStefano & Morgan,
2014; Garrido et al., 2015; Koziol, 2010; Li, 2014; Yang & Xia, 2015; Yu, 2002) on robust
model-fit indexes for ordered categorical data reveals three limitations. First, previous simulation
studies on WLSMV and ULSMV were implemented with small to moderate sample sizes
(mostly less than 1,000 observations). It is unknown whether the patterns found in the previous
studies are due to the asymptotic properties of the estimators or simply to sample fluctuation.
Because RMSEA, CFI, and TLI are all defined in the population as functions of the asymptotic
fit-function value (Bentler, 1990; Bentler & Bonett, 1980; Cudeck & Browne, 1992; Steiger,
1990; Steiger & Lind, 1980; Tucker & Lewis, 1973), it is important to ensure that the fit indexes
in the population are reflections of model-data fit, rather than being confounded with the
thresholds for categorization when the threshold structure is not misspecified for the analysis
model. Second, the PR fit indexes are calculated without much theoretical justification, and they
do not converge to the original definitions of RMSEA, CFI, and TLI. It becomes important to
evaluate their SR counterparts for WLSMV and ULSMV. Third, the recent literature suggests to
apply ULSMV unless it results in nonconvergent solutions (e.g., Forero et al., 2009), but its
model-fit indexes are rarely explored.
In sum, the present study aims to investigate the PR and SR fit indexes, for the WLSMV
and ULSMV estimators. Their performances are investigated under extremely large and small to
moderate sample sizes. Specifically, the following research questions will be addressed.
7
First, to what degree do the PR and SR fit indexes for WLSMV and ULSMV depend on
the thresholds for categorization under extremely large samples? I expect that the PR and SR fit
indexes from WLSMV and the PR fit indexes from ULSMV depend on the threshold values,
given that the misspecification only lies in the factor structures rather than the thresholds. Results
from this part of my study will enhance practitioners’ understanding: if PR and SR fit indexes
from WLSMV and the PR fit indexes from ULSMV are dependent on the threshold values, such
dependency is the property of the estimators and correction factors rather than sample
fluctuation, and therefore the cutoff values need to be threshold specific. I also expect that the SR
fit indexes from ULSMV under extremely large samples are irrelevant to the magnitudes of the
threshold values.
Second, under small to moderate sample sizes, are the SR fit indexes better
approximations of their defined values compared with the PR fit indexes and the uncorrected
model-fit indexes without any robust correction? The reason to include the uncorrected model-fit
indexes is that they share the same asymptotic values as the SR fit indexes (i.e., the original
definitions of RMSEA, CFI, and TLI), but their small-sample values can be different. Because
the SR fit indexes converge to the definitions of RMSEA, CFI, and TLI and have their values
adjusted at the sample level, I expect that the SR fit indexes are the most accurate estimates of
their corresponding defined values compared with the uncorrected and PR fit indexes. In
addition, based on the first research question, I expect that the uncorrected, PR, and SR fit
indexes from WLSMV and the PR fit indexes from ULSMV will all be influenced by the
thresholds given small to moderate sample sizes. The SR fit indexes from ULSMV will be
relatively stable across threshold conditions under finite samples.
Third, if the continuous data before categorization are analyzed using ML, to what degree
are the RMSEA, CFI, and TLI different from the corresponding model fit indexes based on
WLSMV and ULSMV after categorization? The conventional cutoff values (e.g., Hu & Bentler,
1999) developed under continuous data analyzed using ML have been applied and cited in the
majority of the studies where WLSMV or ULSMV have been used. This practice is problematic
if the model-fit indexes from WLSMV and ULSMV are largely different from ML. Because
WLSMV and ULSMV use different fit functions from ML, I expect that the model fit indexes
based on WLSMV and ULSMV will be different from those based on ML.
8
Answering the above research questions is critical for the practice of SEM with ordered
categorical data. Currently, Mplus calculates RMSEAPR, CFIPR, and TLIPR for WLSMV and
ULSMV by default. Previous simulation studies (Beauducel & Herzberg, 2006; DiStefano &
Morgan, 2014; Garrido et al., 2015; Koziol, 2010; Li, 2014; Yang & Xia, 2015; Yu, 2002)
investigated only the PR fit indexes. Despite the conclusions that the PR fit indexes from
WLSMV vary by conditions and the conventional cutoff values should be applied with caution,
practitioners still use them and cite the conventional cutoff rules (e.g., Currier & Holland, 2014;
Eaton et al., 2013; Keyes et al., 2013). This dissertations aims to provide a clearer view of the
use of RMSEA, CFI, and TLI for WLSMV and ULSMV. The SR fit indexes are investigated as
alternatives to the PR fit indexes in both extremely large and small to moderate samples. To
address the above research questions, a simulation study was conducted, with various sample
sizes, types of model misspecifications, numbers of categories, and thresholds for categorization.
In the next chapter, a detailed literature review on the robust model-fit indexes for
WLSMV and ULSMV is provided. Chapter 3 presents the design of the simulation study,
followed by the results shown in Chapter 4. Discussions and conclusions are given in Chapter 5.
9
CHAPTER 2
REVIEW OF LITERATURE
Confirmatory Factor Analysis with Ordered Categorical Variables
Confirmatory Factor Analysis Model
Educational and behavioral research frequently involves latent variables that are not
directly observed, such as motivation, anxiety, and depression. Researchers develop a set of
items to measure such latent variables indirectly. The confirmatory factor analysis (CFA) model
is a major technique in the framework of SEM. It is used to examine how well latent factors are
measured by observed items, and how much the latent factors correlate with each other. To
investigate the performances of the robust model-fit indexes, the current dissertation will employ
the CFA model in the simulation study.
Let m represent the number of observed variables and q be the number of latent factors. A
typical confirmatory factor analysis (CFA) model is expressed as
     , (1)
where  is the m × m population variance-covariance matrix for the observed variables, Λ is
the m × q loading matrix,  is the q × q variance-covariance matrix among the factors, and  is
the m × m residual variance-covariance matrix. Let θ be the vector containing model parameters.
A CFA model estimates the loading parameters in Λ , variances and covariances among the
latent factors in  , and residual variances and covariances in  . Some parameters are typically
fixed. For example, the factor variances in  can be constrained to 1 for identification purposes,
and the residual covariances in  are constrained to 0 when error scores across items are
assumed uncorrelated.
One of the purposes of SEM analysis is to test whether the hypothesized model fits the
sample data. A fit function is used to quantify the discrepancy between the observed variance-
covariance matrix and the model-implied variance-covariance matrix. Let S represent the sample
covariance matrix and (θ) be the model-implied covariance matrix governed by θ . Also let s
and σ(θ) be the vectorizations of the non-duplicated elements in S and (θ) , respectively. The fit
function, F, is defined as
10

F   s  σ  θ   W1  s  σ  θ   , (2)
where W is a positive definite weight matrix. The model parameters in θ are estimated by
minimizing the fit function. Let F̂ be the minimized fit function from a sample of n
observations. The T statistic computed as T  (n  1) Fˆ follows a central chi-square distribution
with the expectation being the model degrees of freedom, assuming that data follow a
multivariate normal distribution and the model is correctly specified (e.g., Bollen, 1989). The T
statistic is labeled as the chi-square statistic in the output of SEM software programs (e.g.,
Mplus). The chi-square statistic is thus compared against the target chi-square distribution to test
the null hypothesis that the unstructured covariance matrix equals to the model-implied
covariance matrix.
Previous researchers have proposed different methods to estimate the parameters in θ ,
depending on what weight matrix W is used. One of the most frequently applied estimators is
ML (e.g., Bollen, 1989), where W is a function of the model-implied covariance matrix (θˆ ) .
For ordered categorical data, the measures of association, model parameters, and the weight
matrix W are different from those using ML, as detailed below.
Threshold and Polychoric Correlation
When data are ordered and categorical, the measures of association are different from
those for continuous variables. One common assumption for ordered categorical variables is that
a continuous variable underlies an ordered categorical variable. The unobserved continuous
variable is categorized into the observed ordinal variable by applying a set of thresholds. When
the observed variables are binary, the correlation between the two underlying continuous
variables is called tetrachoric correlation. When the observed variables have more than two
categories, the corresponding correlation is named polychoric correlation. In this dissertation, the
term polychoric correlation is used for variables with any number of ordered categories,
recognizing that the tetrachoric correlation is a special case.
It is known that when a Pearson product-moment correlation coefficient for two observed
ordered categorical variables is calculated as a measure of association, the resulting correlation
coefficient is an underestimate of the relationship between the underlying continuous variables
(e.g., Olsson, 1979). Olsson (1979) proposed the one-step maximum likelihood estimation
11
method and the two-step maximum likelihood method to estimate the polychoric correlation
coefficient. Because the two-step ML method is commonly applied to estimate the polychoric
correlation coefficient in the current literature and it is implemented in Mplus when WLSMV or
ULSMV is applied, the rest of this section introduces the two-step ML technique.
Similar notations to Olsson (1979) are used for demonstration purposes. Let x and y be
two observed ordered categorical variables, and  and  be their corresponding underlying
continuous variables, respectively. The joint distribution of  and  is bivariate normal with  *
being the polychoric correlation. Suppose x has s categories and y has r categories. x and y are
then obtained by categorizing  and  using a set of thresholds, such that
x = i, if ai 1    ai
and
y = j, if b j 1    b j ,
where i can be 1, 2…, s, j can be 1, 2…, r, and ai and b j are the thresholds for categorization
(note that a0  b0   and as  br   ).
For the two-step ML estimation method (Olsson, 1979), the first step is to estimate the
thresholds from the cumulative marginal proportions. Let  be the univariate standard normal
cumulative distribution function. The thresholds are estimated as

ai  11  Pi 
and
b j  11 ( P j ) ,
where Pi and P j are the observed cumulative marginal proportions. The second step is to
estimate the polychoric correlation coefficient. Let  ij be the probability corresponding to the
observed scores of x = i and y = j. The sample likelihood can then be expressed as

s r
L  C    ij ij ,
n
i j
where C is a constant and nij is the number of observations that have x = i and y = j. The log-
likelihood is then
12
s r
l  ln L  ln C   nij ln  ij .
i 1 j 1
Let  2 be the bivariate standard normal cumulative distribution function with correlation  * ,
and 2 be the corresponding probability density function. It follows that
 ij  2  ai , b j   2  ai 1 , b j   2  ai , b j 1   2  ai 1 , b j 1  ,
and that
 ij
 2  ai , b j   2  ai 1 , b j   2  ai , b j 1   2  ai 1 , b j 1  .
 *
Solve the following equation to get the estimate of  * :
l

2  ai , b j   2  ai 1 , b j   2  ai , b j 1   2  ai 1 , b j 1   0 .
s r n
 
ij
(3)
 * i 1 j 1  ij
In equation (3), the only unknown quantity is  * because the first step has provided the estimates
for the thresholds. An iterative method, for example, the Newton-Raphson algorithm, can thus be
applied to find the approximation of  * . As mentioned by Olsson (1979), the two-step
estimation method is not theoretically optimal, but the differences between the two-step
estimation method and the one-step ML procedure are small.
Estimation Methods for CFA with Ordered Categorical Data
Muthén (1978) used the following fit function for the weighted least squares (WLS)
estimator:
FWLS   s  σ  θ   W1  s  σ  θ   . (4)
The differences between equations (2) and (4) are that σ(θ) in equation (4) is a vector containing
the thresholds and the polychoric correlations, s contains the corresponding sample estimates,
and the weight matrix W in (4) is the asymptotic variance-covariance matrix of the thresholds
and polychoric correlations (Olsson, 1979; Jöreskog, 1994). A typical CFA model with ordered
categorical data has a θ vector including the thresholds for categorization, loading parameters in
Λ , variances and covariances among the latent factors in  , and covariances in  . Some
parameters are typically fixed; for example, the factor variances in  (constrained to 1) and the
13
residual covariances in  (constrained to 0). Every residual variance in  is fixed to a value
such that the variance of the underlying continuous variable is always 1.
Let Γ be the asymptotic covariance matrix of the elements in s, and Γ̂ be its sample
ˆ . Muthén (1993) brought up the concern that W  Γ
estimate. For the WLS estimator, W  Γ ˆ
can result in statistical and computational problems because Γ̂ can be very large when the
model is complex and the number of ordered categorical variables is large. The estimation of a
large dimension of Γ̂ based on small to moderate sample sizes can therefore be very inaccurate.
Second, when inverting Γ̂ , the computational burden increases substantially with the increase in
the dimension. In addition, Γ̂ is very likely to be singular for small sample sizes with some score
patterns having very low or high probabilities (Muthén et al., 1997).
Based on a Taylor expansion, the asymptotic covariance matrix for the sample estimate
of θ can be shown as (Muthén, 1993)
   
1 1
aV  θ   n1 ΔW1Δ ΔW1ΓW1Δ ΔW1Δ , (5)
where Δ  σ  θ  / θ . Because in equation (5), W doesn’t have to be Γ , Muthén (1993)
suggested using a simpler weight matrix for W so that the inversion of Γ in equation (4) can be
avoided. Muthén (1993) suggested W = I or W  diag  Γ  . The former results in the unweighted
least squares (ULS) estimator and the latter leads to the DWLS estimator (Muthén & Muthén,
1998-2015). Let FDWLS and FULS denote the fit functions for the DWLS and ULS estimators,
respectively. The parameter estimates are obtained by searching a vector of θ that minimizes
FDWLS or FULS.
Robust Corrections for the DWLS and ULS Chi-square Statistics
For the WLS estimator (Muthén, 1978), the chi-square statistic is calculated as
TWLS   n  1 FWLS .
It asymptotically follows the central chi-square distribution with its expectation equal to the
model degrees of freedom under correctly specified models. For DWLS, because W only utilizes
the diagonal elements of Γ , the uncorrected chi-square statistic
TDWLS   n  1 FDWLS
14
no longer follows the target chi-square distribution asymptotically, where FDWLS is the fit
function with W  diag  Γ  . Similar to Satorra (1992) and Satorra and Bentler’s (1994) robust
correction for the chi-square statistic, TDWLS can be corrected so that for WLSMV,
TWLSMV  aDWLS   n  1 FDWLS  bDWLS , (6)
where aDWLS and bDWLS are the scale factor and shift factor, respectively, such that

aDWLS  d / tr  U DWLS Γ 
2
,

bDWLS  d  d  tr  U DWLSΓ  / tr  U DWLSΓ 
2 2
,
with
 
UDWLS  W1  W1ΔDWLS ΔDWLS W1ΔDWLS 
1
ΔDWLS W1 . 
Here, d is the model degrees of freedom. TWLSMV thus has its mean and variance both equal to the
mean (i.e., d) and variance (i.e., 2d), respectively, of the target chi-square distribution
asymptotically (Asparouhov & Muthén, 2010). WLSMV has become the most applied method
for analyzing ordered categorical data in the SEM literature (Savalei & Rhemtulla, 2013).
For ULSMV, TULS can be similarly corrected so that the robust chi-square statistic is
TULSMV  aULS   n  1 FULS  bULS , (7)
where FULS is the fit function with an identity weight matrix,

aULS  d / tr  U ULSΓ 
2
,

bULS  d  d  tr  U ULSΓ  / tr  U ULSΓ 
2 2
,
and
 
U ULS  W1  W1ΔULS ΔULS W1Δ ULS 
1
ΔULS W 1 . 
TULSMV thus also has its mean and variance equal to the mean (i.e., d) and variance (i.e., 2d),
respectively, of the target chi-square distribution asymptotically (Asparouhov & Muthén, 2010).
Previous studies (Flora & Curran, 2004; Muthén et al., 1997; Oranje, 2003; Yang-
Wallentin et al., 2010) showed that WLSMV in general outperformed ML and WLS, and the
superiority of WLSMV occurs especially under conditions with small sizes and less than four
15
ordered categories. These studies focused on parameter and standard error estimations. With
respect to the comparison between WLSMV and ULSMV, Forero et al. (2009) investigated
WLSMV and ULSMV using factor analysis models, and concluded that ULSMV provided more
accurate and less variable parameter estimates, more accurate standard errors, and better
coverage rates. Yang-Wallentin et al. (2010) compared the mean-adjusted ULS (ULSM), mean-
adjusted WLS (WLSM), and robust ML based on polychoric correlation matrix. They found
small differences between the ULSM, WLSM, and robust ML methods across different
conditions. Savalei and Rhemtulla (2013) compared the robust chi-square statistic between
WLSMV and ULSMV and recommended ULSMV, particularly in small to medium sample
sizes. Li (2014) further compared the robust ML, WLSMV, and ULSMV in structural regression
models with ordered categorical data. He found that, in general, ULSMV produces more accurate
estimates of standard errors of factor loadings than WLSMV. In sum, ULSMV is found to
perform slightly better than WLSMV in parameter and standard error estimations. Forero et al.
(2009) recommended the use of ULSMV instead of WLSMV unless ULSMV results in a non-
convergent solution.
Robust Model-fit Indexes for WLSMV and ULSMV
Chi-square-based Model-fit Indexes
The chi-square statistic is used to test the null hypothesis that the model explains the data
in the population, which is reported in almost every SEM application. However, the chi-square
statistic only follows the central chi-square distribution under correctly specified models. For
misspecified models, its performance can be problematic. When the sample size increases, the
chi-square statistic will inflate even when the model is only slightly misspecified. Because
hypothesized models are always misspecified in reality, where researchers are looking for a
parsimonious model that fits the data as much as possible, the application of the chi-square
statistic is not as meaningful as it intends to be.
Along with the chi-square statistic, many model-fit indexes have been proposed to
evaluate the model-data fit. RMSEA, CFI, and TLI are three widely applied model-fit indexes in
the SEM literature. RMSEA, CFI, and TLI are all defined as functions of the fit functions in the
population, and thus also functions of the model chi-square statistic.
16
RMSEA quantifies how far the hypothesized model is from the perfect fit to the data.
Therefore, RMSEA is referred to as an absolute measure of the model-data fit (McDonald & Ho,
2002). Let F represent a general fit function, which can be either FDWLS or FULS. Let H be the
hypothesized model and FH and dH represent the fit function and degrees of freedom from the
hypothesized model, respectively. Similarly, B denotes the baseline model. The population
RMSEA (RMSEApop), is defined as
FH
RMSEA pop  . (8)
dH
Conceptually, RMSEA measures the degree of misfit per model degree of freedom. Given a
sample size of n, the sample estimate of RMSEAn is calculated as
  n  1 FˆH  d H 
RMSEA n  max  0, . (9)

  n  1  d H


CFI and TLI compare the fit of the hypothesized model with the baseline model. CFI and
TLI are therefore referred to as incremental fit indexes (McDonald & Ho, 2002). The population
CFI and TLI are defined as
FH
CFIpop  1  (10)
FB
and
FH / d H
TLIpop  1  . (11)
FB / d B
Because the baseline model yields the worst model-data fit, CFI and TLI represent how much the
fit and the fit per degree of freedom, respectively, improve when comparing the hypothesized
model with the baseline model. The sample estimates of CFIpop and TLIpop are calculated as
 n  1 FˆH  d H
CFIn  1  (12)
 n  1 FˆB  dB
and
TLIn  1 
 n  1 FˆH  dH  dB
. (13)
 n  1 FˆB  dB dH
17
Population-corrected Robust Model-fit Indexes
Because RMSEA, CFI, and TLI are all functions of the chi-square statistic given finite
sample sizes, when WLSMV or ULSMV are applied, it is conceptually necessary to replace the
uncorrected chi-square statistic with the robust chi-square statistic. The model-fit indexes
calculated in such a fashion are called population-corrected (PR) model-fit indexes (Brosseau-
Liard et al., 2012), and are labeled as RMSEAPR, CFIPR, and TLIPR.
Let T, d , a, and b be the robust chi-square statistic, the model degrees of freedom, the
scale factor, and the shift factor, respectively, for either WLSMV or ULSMV. The PR model-fit
indexes given a sample size n are calculated as
 a  n  1 FˆH  bH  d H 
RMSEA PR, n  max  0, H , (14)

  n  1 dH 

aH  n  1 FˆH  bH  d H
CFIPR, n  1  , (15)
a  n  1 Fˆ  b  d
B B B B
and
aH  n  1 FˆH  bH  d H d B
TLIPR, n  1   . (16)
aB  n  1 FˆB  bB  d B d H
Comparing equations (14), (15), and (16) with (9), (12), and (13), the PR model-fit indexes are
calculated simply by replacing TH  (n  1) FˆH with TH  aH (n  1) FˆH  bH for the hypothesized
model, and TB  (n  1) FˆB with TB  aB (n  1) FˆB  bB for the baseline model.
The PR model-fit indexes are calculated by current software programs (e.g, Mplus) and
are reported in almost every study in substantive areas which applies WLSMV as the estimation
method. However, such a robust correction for the model-fit indexes is not theoretically
grounded. For continuous data analyzed by the ML estimator with Satorra and Bentler’s robust
correction, Brosseau-Liard et al. (2012) and Brosseau-Liard and Savalei (2014) pointed out that
the PR model-fit indexes are problematic because they do not converge to their population
definitions when the model is misspecified. Because WLSMV and ULSMV correct the chi-
square statistic using logic similar to that behind Satorra and Bentler’s (1994) robust chi-square,
one may expect that the PR model-fit indexes based on WLSMV and ULSMV also suffer from
18
the same problem brought up by Brosseau-Liard et al. (2012) and Brosseau-Liard and Savalei
(2014).
Specifically, based on equations (14), (15), and (16), when the model is correctly
specified, RMSEAPR, CFIPR, and TLIPR will converge to their population values defined in (8),
(10), and (11) (i.e., 0, 1, and 1), respectively, as the sample size increases to infinity. However,
when the model is misspecified, the PR model-fit indexes in (14), (15), and (16) will converge to
aH FH
RMSEA PR, +  , (17)
dH
aH FH
CFIPR, +  1  , (18)
aB FB
and
aH FH / d H
TLIPR, +  1  . (19)
aB FB / d B
respectively, which do not agree with their population definitions shown in (8), (10), and (11).
Specifically, for RMSEAPR in (14), when n goes to infinity, bH  d H becomes negligible
compared with aH  n  1 F H , and therefore RMSEAPR converges to
aH (n  1) FH / ((n  1)dH )  aH FH / dH , which is different from FH / d H . Similarly, CFIPR
converges to 1  aH (n 1) FH / (aB (n 1) FB )  1  aH FH / (aB FB ) and TLIPR converges to
1  aH (n  1) FH dB / (aB (n 1) FBdH )  1  aH FH dB / (aB FBdH ) .
Sample-corrected Robust Model-fit Indexes
In an unpublished article by Li and Bentler (2006), an alternative method was presented

to calculate the robust RMSEA when the Satorra and Bentler’s (1994) robust method is applied
for continuous variables with the ML estimator. The resulting RMSEA is named as the SR
RMSEA (Brosseau-Liard et al., 2012), labeled here as RMSEASR. Brosseau-Liard and Savalei
(2014) applied Li and Bentler’s (2006) method to CFI and TLI and labeled them as CFISR and
TLISR. Brosseau-Liard et al. (2012) and Brosseau-Liard and Savalei (2014) showed that as the
sample size increases to infinity, RMSEASR, CFISR, and TLISR converge to the population
definitions of RMSEA, CFI, and TLI, but RMSEAPR, CFIPR, and TLIPR do not. Xia, Yung, and
19
Zhang (in press) included the SR and PR fit indexes in a simulation study, and found that the SR
fit indexes are more accurate estimates of the population values than the PR fit indexes under
varied levels of nonnormality, model misspecifications, and sample sizes. However, because Li
and Bentler’s (2006) article does not focus on the SR model-fit indexes, and the current software
programs such as Mplus, EQS (Bentler, 2008), and the CALIS procedure of SAS/STAT 14.1
(SAS Institute Inc., 2015) report the PR model-fit indexes only, the SR model-fit indexes have
been rarely applied.
Although the SR fit indexes are proposed under continuous data with Satorra and
Bentler’s (1994) robust corrections, they can be similarly calculated for WLSM and WLSMV.
Following Li and Bentler (2006), Brosseau-Liard et al. (2012), and Brosseau-Liard and Savalei
(2014), the SR fit indexes utilize the robust chi-squares while at the same time converge to the
population definitions. When WLSMV or ULSMV is applied, the SR model-fit indexes can be
computed as:
  n  1 FˆH   d H  bH  / aH 
RMSEASR, n  max  0, , (20)

  n  1 dH 

CFISR, n  1 
 n  1 FˆH   dH  bH  / aH ,
(21)
 n  1 FˆB   dB  bB  / aB
and
TLISR, n  1 
 n  1 FˆH   dH  bH  / aH  dB .
(22)
 n  1 FˆB   dB  bB  / aB dH
RMSEASR, CFISR, and TLISR converge to their population definitions in equations (8), (10), and
(11), respectively. Because the robust model-fit indexes in equations (20), (21), and (22)
incorporate the robust chi-squares at the sample level, with their corresponding population
definitions unchanged, they are named as the SR fit indexes (Brosseau-Liard et al., 2012).
Previous Studies on the Robust Model-fit Indexes for WLSMV and ULSMV
Because the SR mode fit indexes haven’t been made available in any SEM software
program, and their application hasn’t been extended to WLSMV and ULSMV, previous
simulation studies regarding the robust model-fit indexes in WLSMV calculated the PR model-
fit indexes, although none of them mentioned it explicitly. In addition, the ULSMV robust
20
model-fit indexes are rarely investigated. This section therefore provides a brief literature review
on the performances of the PR model-fit indexes for WLSMV.
Yu (2002) conducted a simulation study to investigate the adequacy of the conventional
cutoff criteria (i.e., RMSEA < .06, TLI > .95, CFI > .96; Hu & Bentler, 1999) for several
goodness-of-fit indexes in WLSMV for CFA models, including RMSEAPR, TLIPR, and CFIPR.
Yu varied the sample sizes, types of model misspecification, and types of variables (i.e., binary,
normal, and nonnormal data). Yu found that the mean RMSEAPR based on WLSMV for binary
data was lower than the mean RMSEA based on ML for continuous normal data, and the mean
TLI and mean CFI were greater than those for ML. Yu stated that the best choices of the cutoff
values actually depended on models, and no universal cutoff values could be established for
WLSMV. Yu also suggested practitioners implement a simulation study using their target model
to decide what appropriate cutoff values should be applied.
Beauducel and Herzberg (2006) compared ML based on Pearson correlation and
WLSMV for CFA with ordered categorical data under correctly specified models. They varied
the numbers of factors, numbers of observed variables, numbers of categories, and sample sizes.
They evaluated the chi-square statistic, item loading estimations, as well as the RMSEAPR,
CFIPR, and TLIPR. They found that in general the WLSMV chi-square statistic outperformed the
ML chi-square regarding model rejection rates, especially under the conditions with two or three
ordered categories. Regarding the fit indexes, they claimed that the behavior of the PR model-fit
indexes in WLSMV wasn’t very clear, and different cutoff criteria may be used to evaluate
model data fit. The findings from Beauducel and Herzberg (2006) were consistent with those
from Yu (2002), supporting the argument from Marsh, Hau, and Wen (2004) that the
development of the “golden rules” of model fit may be problematic.
Koziol (2010) implemented a simulation study under the measurement invariance
framework. Koziol investigated the effects of categorization and censoring on the chi-square test
statistic, RMSEAPR, and CFIPR with respect to power and type I error rates. ML, Yuan-Bentler
scaled ML (Yuan & Bentler, 2000), and WLSMV were used to analyze the ordered categorical
data in Koziol’s (2010) study. Koziol used RMSEAPR > .06 and CFIPR < .95 (Hu & Bentler,
1999) to evaluate the power and type I error rates for all conditions. Koziol found that the power
of WLSMV RMSEAPR and CFIPR increased with larger sample sizes and less complex models.
21
Koziol concluded that the WLSMV RMSEAPR and CFIPR outperformed the ML RMSEA when
the number of categories was four or less than four.
Li (2014) compared MLR, USLMV, and WLSMV for ordered categorical data with
respect to parameter estimates, standard errors, and RMSEAPR. In this study, only correctly
specified models were considered. Li found that the WLSMV RMSEAPR performed well.
However, RMSEAPR tended to be positively biased with finite sample sizes. As the sample size
increased, the means of RMSEAPR approached 0, regardless of the numbers of observed
categories and thresholds used for categorization. Li (2014) also compared the ULSMV
RMSEAPR and found that it was similar to the WLSMV RMSEAPR.
Sass, Schmitt, and Marsh (2014) evaluated the WLSMV chi-square statistic and model fit
indexes with ordered categorical data within a measurement invariance framework. They used
the changes of RMSEAPR, CFIPR, and TLIPR for model comparison and concluded that, for
WLSMV, researchers should not use the change of those model-fit indexes, especially under
misspecified models, because they depend on several design factors such as sample size and
model complexity.
DiStefano and Morgan (2014) compared DWLS methods for ordinal data including
DWLS in LISREL (Jöreskog & Sörbom, 1996), and WLSM and WLSMV in Mplus. They
investigated RMSEAPR and CFIPR under correctly specified models, and found that the
RMSEAPR and CFIPR were not sensitive to sample sizes, thresholds for categorization, numbers
ordered categories, and degrees of nonnormality for the underlying continuous variables.
Using exploratory factor analysis models, Yang and Xia (2015) examined the
performance of the WLSMV robust chi-square statistic, WLSMV fit indexes (RMSEAPR, CFIPR,
and the standardized root mean square residual, SRMR), eigenvalue > 1 rule (Kaiser, 1960), and
parallel analysis in determining the number of factors to retain for ordered categorical data. They
found that the fit indexes led to acceptable decisions when the number of scale points was four,
the number of items was large, sample size was at least 200, and the thresholds for categorization
were similar across items. However, they showed that the PR model fit indexes might perform
poorly under two-point scales and short scales, and suggested that RMSEA < .06 (Hu & Bentler,
1999) should not be universally applied, especially for two-point scales and short scales. They
also concluded that CFIPR performed either similarly to or slightly worse than RMSEAPR.
22
Garrido et al. (2015) further systematically evaluated the performance of the WLSMV
RMSEAPR, CFIPR, TLIPR, and SRMR in exploratory factor analysis models in the estimation of
the number of factors with ordered categorical variables. They found that CFIPR and TLIPR led to
nearly identical conclusions and were the most accurate model-fit indexes compared with
RMSEAPR and SRMR. They also brought up the concern that it was difficult to develop a set of
cutoff values for the fit indexes. In addition, they claimed that RMSEAPR tended to accept highly
misspecified models when the observed variables had large unique variances, but CFIPR and
TLIPR were less influenced by the magnitudes of the unique variances.
Summarizing the above studies leads to two conclusions. First, under correctly specified
models, the WLSMV RMSEAPR approaches 0 as the sample size increases, and both RMSEAPR
and CFIPR are insensitive to the other design factors such as the number of latent factors, number
of observed items, and threshold for categorization (DiStefano & Morgan, 2014). The results for
correctly specified models are consistent with the formula of the DWLS. Asymptotically, the fit
functions equal to 0 when the model is correctly specified. Therefore, the WLSMV RMSEAPR,
CFIPR, and TLIPR shown in equations (14), (15), and (16) will also converge to 0, 1, and 1,
respectively, as the sample sizes increase to infinity, and the asymptotic values of RMSEAPR,
CFIPR, and TLIPR under correctly specified models do not depend on the threshold values.
Second, when models are misspecified, the patterns of the WLSMV RMSEAPR, CFIPR,
and TLIPR are not clear (e.g., Yu, 2002), and they depend on other design factors such as the
thresholds for categorization and types of models (Garrido et al., 2015; Koziol, 2010; Yang &
Xia, 2015; Yu, 2002). The possible reasons for the dependency of the WLSMV PR fit indexes be
twofold. According to equations (14), (15), and (16), RMSEAPR, CFIPR, and TLIPR converge to
distorted values when the sample size goes to infinity under misspecified models. The large
sample property of RMSEAPR, CFIPR, and TLIPR are therefore rather unpredictable, and depend
on the scale and shift factors. In addition, because the diagonal weight matrix employed by
WLSMV contains the asymptotic variances of the thresholds and polychoric correlations, one
may expect that the fit function, and thus the robust model-fit indexes are influenced by the
thresholds under misspecified models. Therefore, even if a set of robust RMSEA, CFI, and TLI
can be found to converge to the population definitions using the WLSMV fit function, their
asymptotic values depend on the thresholds for categorization. Furthermore, the above studies on
23
the PR model-fit indexes focused on finite sample sizes. The asymptotic behavior of the PR
model-fit indexes remains unexplained.
Purposes of Study
As summarized above, previous studies (e.g., Sass et al., 2014; Yang & Xia, 2015; Yu,
2002) found that the WLSMV PR fit indexes behave rather unpredictably under misspecified
models. Researchers in substantive areas can find it difficult to evaluate the overall model-data
fit when ordered categorical data are analyzed. This dissertation study is driven by the fact that
the current practice is to apply and report the WLSMV PR model fit indexes, as well as to use
the conventional rules, despite the arguments of their application from the previous research
(e.g., Marsh et al., 2004).
The main purposes of the current research are threefold. The previous simulation studies
reviewed above are all using small to moderate sample sizes. Because RMSEA, CFI, and TLI are
defined as functions of the population fit function, it is important to investigate if their
asymptotic values are appropriate measures of model-data fit rather than functions of the
thresholds. The first purpose is to show the dependency of RMSEA, CFI, and TLI on the
thresholds under extremely large samples for both WLSMV and ULSMV estimators when the
model is misspecified. The dependency of model fit indexes on the thresholds is not a desirable
property, because when the misspecification lies within the factor structures (e.g., under-
specification of the number of factors or missing cross-loadings), model fit indexes should only
measure the model-data discrepancy pertaining to factor structures rather than be confounded
with the threshold values. In addition, if the model fit indexes depend on the thresholds, their
corresponding cutoff values should also be threshold specific, which is impractical.
Second, this dissertation aims to generalize the SR model fit indexes (Brosseau-Liard et
al., 2012; Brosseau-Liard and Savalei, 2014; Li & Bentler, 2006) to WLSMV and ULSMV
estimators as alternatives, and compare them with their PR counterparts. The uncorrected model-
fit indexes are also included for comparison purposes because they share the same asymptotic
values as the SR fit indexes. The small-sample values of the uncorrected indexes and the SR fit
indexes can be different.
The third purpose is to compare the model fit indexes from ML before the continuous
data are categorized, and the PR and SR fit indexes from ULSMV and ULSMV estimators. The
24
conventional cutoff values (e.g., Hu & Bentler, 1999) would not be valid if the model-fit indexes
from WLSMV and ULSMV are different from ML.
Overall, the present study investigates the PR and SR fit indexes for WLSMV and
ULSMV, under extremely large and small to moderate sample sizes. Specifically, the following
research questions are addressed.
the thresholds for categorization under extremely large samples? I expect that the PR and SR fit
indexes from WLSMV and the PR fit indexes from ULSMV depend on the thresholds, given that
the misspecification only lies in the factor structures rather than the thresholds. I also expect that
the SR fit indexes from ULSMV under extremely large samples are irrelevant to the magnitudes
of the threshold values.
approximations of the definitions of RMSEA, CFI and TLI compared with the PR fit indexes and
the uncorrected model-fit indexes without any robust correction? Because the SR fit indexes
converge to the definitions of RMSEA, CFI, and TLI and have their values adjusted at the
sample level, I expect that the SR fit indexes are the most accurate estimates of their
corresponding defined values compared with the uncorrected and PR fit indexes. In addition,
based on the first research question, I expect that the uncorrected, PR, and SR fit indexes from
WLSMV and the PR fit indexes from ULSMV are all influenced by the thresholds given small to
moderate sample sizes. The SR fit indexes from ULSMV are relatively stable across threshold
conditions under finite samples.
Third, if the continuous data before categorization are analyzed using ML, to what degree
are the RMSEA, CFI, and TLI different from the corresponding model fit indexes based on
WLSMV and ULSMV after categorization? Because their fit functions are different, I expect that
their values are different as well, consistent with Yu’s research (2002).
25
CHAPTER 3
METHODS
A simulation study was conducted to answer the three research questions laid out in the
previous chapter. This chapter described the details of the simulation design, procedure for data
generation and analysis, and the assessment of outcome variables.
Simulation at the Population Level
To approximate the asymptotic values of RMSEA, CFI, and TLI for each condition as
described below, an extremely large sample size, n = 1,000,000, was used.
Factor Structure
Two CFA models were employed for data generation.

Model 1. Model 1 was a CFA model with 3 factors and no cross-loading. The number of
items was varied at 9 and 18, and the factor correlation  was varied at .3 and .7.
When the model had 9 items, the population variance-covariance matrix Σ used to
generate the continuous data was shown in equation (1). To repeat,
     ,
where
.7 .7 .7 0 0 0 0 0 0 
   0 0 0 .7 .7 .7 0 0 0  ,
 0 0 0 0 0 0 .7 .7 .7
1  
    1   ,
   1 
and
diag     .51, .51, .51, .51, .51, .51, .51, .51, .51 .
When the model had 18 items,
26
.7 .7 .7 .7 .7 .7 0 0 0 0 0 0 0 0 0 0 0 0 
   0 0 0 0 0 0 .7 .7 .7 .7 .7 .7 0 0 0 0 0 0  ,
 0 0 0 0 0 0 0 0 0 0 0 0 .7 .7 .7 .7 .7 .7
1  
    1   ,
   1 
and
.51, .51, .51, .51, .51, .51, .51, .51, .51, .51, .51, 
diag      .
.51, .51, .51, .51, .51, .51, .51 
In words, Model 1 had the loading parameters equal to .7, factor variances equal to 1, factor
covariances (i.e., correlations in this case) equal to .3 or .7, and residual variances equal to .51.
Model 2. Model 2 was also a CFA model, but some of the items had loadings on two
factors. When the number of items was 9,
.7 .7 .7 .3 0 0 0 0 .3
   0 0 .3 .7 .7 .7 .3 0 0  ,
.3 0 0 0 0 .3 .7 .7 .7
 1 .3 .3
  .3 1 .3 ,
.3 .3 1 
and
diag     .294, .51, .294, .294, .51, .294, .294, .51, .294 .
When the number of items was 18,
.7 .7 .7 .7 .7 .7 .3 0 0 0 0 0 0 0 0 0 0 .3
   0 0 0 0 0 .3 .7 .7 .7 .7 .7 .7 .3 0 0 0 0 0  ,
.3 0 0 0 0 0 0 0 0 0 0 .3 .7 .7 .7 .7 .7 .7
 1 .3 .3
  .3 1 .3 ,
.3 .3 1 
and
27
.294, .51, .51, .51, .51, .294, .294, .51, .51, .51, .51, 
diag      .
.294, .294, .51, .51, .51, .51, .294 
The loadings of .3 in  represented the cross-loadings. The factor correlation was fixed at .3 for
Model 2. In the misspecified model, the cross-loadings were removed, as described later.
Data Distribution
Continuous data were first generated using a multivariate normal distribution with means
equal to 0 and the variance-covariance matrix as defined above. Thereafter, a set of thresholds
were applied to categorize each continuous variable into an ordered categorical variable. The
number of ordered categories (k) was varied at 2 and 4, which are common in educational and
behavioral research.
 When k = 2, two threshold conditions were used to create ordered categorical data from
the continuous variables: (1) Homogeneous thresholds: all items had the same threshold
of [τ]. (2) Mixed thresholds: the thresholds were {[- τ], [τ], [-τ], [τ], …} such that for
every latent factor, some items had different thresholds from the other items.
When k = 2, τ was varied from 0 to 2 with an interval of .2. When τ = 0, the
homogeneous-threshold and mixed-threshold conditions yielded the same set of
threshold values, and they were thus considered as 1 condition.
 When k = 4, two sets of thresholds were used: (1) Homogeneous thresholds: all items had
the thresholds of [τ - 1, τ, τ + 1]. (2) Mixed thresholds: the odd items had the thresholds
of [τ - 1, τ, τ + 1] and the even items had [-τ - 1, -τ, -τ + 1].
When k = 4, τ was varied from 0 to 1, with an interval of .2. When τ = 0, the
homogeneous-threshold and mixed-threshold conditions were the same. Every threshold
is freely estimated in the model.
In sum, Model 1 led to 128 conditions, resulting from 2 (numbers of items) × 2 (levels of
factor correlation) × 32 (sets of thresholds), and Model 2 led to 64 conditions, from 2 (numbers
of items) × 32 (sets of thresholds). Therefore, the total number of simulation conditions at the
population level was 192.
28
Analysis
Before the ordered categorical data were generated, each data set was analyzed using a
misspecified model with the ML estimator for comparison purposes. After the ordered
categorical data were generated, each generated data set was analyzed using the same
misspecified model. For Model 1, the misspecified model was a single-factor model, with all the
9 or 18 items measuring one latent factor. For Model 2, the misspecified model omitted all cross-
loadings (i.e., the loadings of .3).
WLSMV and ULSMV were applied to estimate the model parameters and evaluate
model-data fit for the ordered categorical data. Only one replication was implemented for each
condition because the sample size was extremely large. The RMSEAPR, CFIPR, and TLIPR and
the corresponding SR model fit indexes were calculated.
The values of the PR and SR model fit indexes for WLSMV and ULSMV were plotted
against the τ values representing the thresholds. This aimed to answer the first research question:
To what degree do the PR and SR fit indexes for WLSMV and ULSMV depend on the
thresholds for categorization under extremely large samples? In addition, the RMSEA, CFI, and
TLI from ML for the continuous data before categorization were also plotted against τ to address
the third research question: To what degree are the RMSEA, CFI, and TLI from ML different
from the corresponding model fit indexes based on WLSMV and ULSMV after categorization?
Simulation at the Sample Level
Simulation Design
The same models and similar design factors described above were used for the simulation
study at the sample level. More specifically, for Model 1, the number of items was varied at 9
and 18, and the factor correlation was varied at .3 and .7. For Model 2, the number of items was
varied at 9 and 18.
The sample sizes were varied at 100, 200, 500, and 1,000. When k = 2, three sets of
thresholds were chosen, such that
{[0], [0], [0], [0], …},
{[1], [1], [1], [1], …},
and
29
{[1], [-1], [1], [-1], [1], …}.
When k = 4, three sets of thresholds were chosen, such that
{[-1, 0, 1], [-1, 0, 1], [-1, 0, 1], …},
{[-.6, .4, 1.4], [-.6, .4, 1.4], [-.6, .4, 1.4], …},
and
{[-.6, .4, 1.4], [-1.4, -.4, .6], [-.6, .4, 1.4], [-1.4, -.4, .6], …}.
For Model 1, the number of conditions was therefore 4 (sample sizes) × 2 (numbers of
items) × 2 (levels of factor correlation) × 2 (numbers of categories) × 3 (sets of thresholds) = 96.
Model 2 resulted in 4 (sample sizes) × 2 (numbers of items) × 2 (numbers of categories) × 3 (sets
of thresholds) = 48 conditions. In sum, the total number of simulation conditions at the sample
level was 144. The number of replicated data sets was 2000 for each condition.
Analysis
Similar to the simulation study at the population level, each continuous data set before
categorization was analyzed using the ML estimator, and the resulting RMSEA, CFI, and TLI
were calculated. After the ordered categorical data were created, each data set was analyzed
using a misspecified model with WLSMV and ULSMV. For each condition, the means and
standard deviations of the PR and SR RMSEA, CFI, and TLI were calculated. In addition, the
uncorrected model fit indexes without any robust correction were calculated for comparison
purposes.
The means of the uncorrected, PR, and SR model fit indexes were compared to their
asymptotic values approximated by the corresponding model with n = 1,000,000. This aimed to
answer the second research question: under small to moderate sample sizes, are the SR fit
indexes better approximations of the definitions of RMSEA, CFI, and TLI compared with the
uncorrected and PR fit indexes? The means of the model fit indexes based on ML were further
compared with the PR and SR fit indexes to address the third research question.
30
CHAPTER 4
RESULTS
Results at the Population Level
The values of each model-fit index were plotted against the τ values under each model,
factor correlation, threshold condition, number of items, and number of ordered categories. This
section only presented the results from Model 1 with 9 items, focusing on whether or not each
index depended on the thresholds characterized by τ, and if the ML-based model fit indexes were
similar to or different from those in WLSMV and ULSMV. Results from the other conditions,
including the conditions for Model 1 with 18 items and all the conditions for Model 2, showed
similar patterns and are shown in Appendix A.
RMSEA
Figure 1 presents the RMSEA values from Model 1 with factor correlations at .3 or .7,
homogeneous and mixed threshold conditions, 9 items, and 2 ordered categories. The values of
the RMSEA from ML before data categorization and the RMSEAPR and RMSEASR based on
WLSMV and ULSMV were plotted against τ.
For WLSMV, Figure 1 shows that both the RMSEAPR and RMSEASR largely depended
on the threshold. For example, when the factor correlation was .3, as τ changed from 0 to 2.0, the
RMSEAPR decreased from .13 to .04, and RMSEASR from .14 to .04, approximately. Given the
same data generation model and the same fitted model, the model-data fit indicated by
RMSEAPR and RMSEASR led to different conclusions depending on the threshold: When τ was
0, RMSEAPR and RMSEASR suggested that the fitted model was unacceptable; when τ increased
to 2.0, they suggested that the hypothesized model was a good model. When the factor
correlation was .7, RMSEAPR and RMSEASR also decreased with increasing τ. However, because
the single-factor model became less misspecified as the factor correlation increased, the
dependency of RMSEAPR and RMSEASR on τ was less severe compared with the conditions with
factor correlation of .3.
For the RMSEAPR from ULSMV, Figure 1 shows that it had very similar values to the
WLSMV RMSEAPR and RMSEASR. The ULSMV RMSEAPR also decreased with increasing τ.
31
When the factor correlation increased from .3 to .7, the ULSMV RMSEAPR became less
dependent on τ because the single-factor model became less misspecified.
For the RMSEASR from ULSMV, Figure 1 shows that it was very stable across the
threshold conditions. This was expected because, as explained in Chapter 2, the fit function of
ULSMV does not include the threshold information in the identity weight matrix, and the
asymptotic value of its RMSEASR doesn’t involve the scale and shift factors.
Regarding the RMSEA from ML, it had discrepant values from the RMSEAPR and
RMSEASR from WLSMV and ULSMV. Based on Figure 1, the conventional cutoff values
resulted in more frequent acceptance of the misspecified models using WLSMV or ULSMV
compared with ML. Therefore the conventional cutoff values developed under ML with
continuous data should not be applied under ordered categorical data.
Figure 2 presents the results of RMSEA for the 4-categories condition. Similarly, the
RMSEAPR and RMSEASR from WLSMV, and the RMSEAPR from ULSMV depended on the
thresholds. As τ increased, they all decreased. However, the dependency on the thresholds
became less severe compared with the 2-category conditions in Figure 1. One reason might be
due to the fact that when the number of categories increases, the data become closer to being
continuous, and thus less influenced by thresholds. The second reason might be that the
thresholds selected for the 4-categories condition in this simulation study led to less extreme
distributions compared with the thresholds for the 2-categories condition. The RMSEASR from
ULSMV was stable across the threshold values. However, because the RMSEA from ML still
had different values from the RMSEAs obtained from the other two estimators, the conventional
cutoff values developed under ML for continuous data should not be applied to ordered
categorical data.
As a summary, only the RMSEASR from ULSMV was independent of the thresholds
across all conditions. The RMSEAPR from ULSMV and WLSMV, and the RMSEASR from
WLSMV were all confounded with the threshold values. The ML RMSEA were different from
the WLSMV RMSEAs and ULSMV RMSEAs, suggesting that the conventional cutoff rules
shouldn’t be applied to ordered categorical data.
32
CFI and TLI
Figure 3 and 4 present the CFI values from Model 1 under the 2- and 4-category
conditions, respectively.
For WLSMV, the CFIPR and CFISR both depended on the thresholds, especially when
data were binary and ρ = .3. For example, in Figure 3, as τ increased from 0 to 2.0 for conditions
with homogeneous thresholds and ρ = .3, the WLSMV CFIPR increased from around .61 to .70,
and the CFISR from .72 to .75. When ρ increased to .7, or the data had 4 categories as shown in
Figure 4, the dependency seemed to be ignorable.
For ULSMV, the CFIPR was also influenced by the threshold value. For instance, when τ
increased from 0 to 2.0 under the 2-category conditions, homogeneous thresholds, and ρ = .3, the
ULSMV CFIPR increased from around .59 to .64. Similarly, when data had 4 categories, and ρ
increased to .7, the dependency decreased. The CFISR from ULSMV was stable across all
threshold conditions, regardless of the size of τ and whether the thresholds were homogeneous or
mixed. As an example, Figure 3 shows that the ULSMV CFISR remained at .71 under both the
homogeneous and mixed threshold conditions, and didn’t change with τ.
Figure 5 and Figure 6 present the TLI values from Model 1 under the 2- and 4-category
conditions, respectively. The TLIs shared similar patterns to the CFIs in Figure 3 and 4. First,
only the TLISR from ULSMV was stable across the threshold conditions. The dependency of the
TLIPR from ULSMV, and the TLIPR and TLISR from WLSMV all decreased when data changed
from binary to 4-category, and when ρ increased from .3 to .7.
Furthermore, Figure 3 to 6 all showed that the CFI and TLI based on ML had different
values from the corresponding indexes from WLSMV and ULSMV, suggesting that the
conventional rules developed under ML with continuous data should not be applied to WLSMV
and ULSMV.
Results at the Sample Level
In this section I present only the results from Model 1 with 9 items. Results from the
other conditions, including the conditions for Model 1 with 18 items and all the conditions for
Model 2, showed similar patterns and are presented in Appendix B. After all data sets were fitted
to the hypothesized models, the replications with inadmissible solutions were excluded from
further analysis. Appendix C presents the inadmissible solutions rates under each condition. The
33
conditions with binary data and asymmetric or mixed thresholds resulted in relatively high
inadmissible solution rates (up to 63.6%) for WLSMV and ULSMV. For the other conditions,
the inadmissible solution rates were mostly within 30%. The general trend was that when the
data had more categories and the sample size increased, the inadmissible solution rates
decreased.
RMSEA
Table 1 presents the means and standard deviations of RMSEA for the conditions of
Model 1 with binary data and 9 items. The results from the 1,000,000 sample were also shown as
approximations of their corresponding asymptotic values.
Consistent with the above results where the sample size was 1,000,000, the RMSEAPR
and RMSEASR from WLSMV and the RMSEAPR from ULSMV varied across the threshold
conditions. For example, the symmetric threshold condition resulted in the WLSMV RMSEASR
= .125, while the asymmetric threshold condition gave a value of .092, given n = 200 and ρ = .3.
Only the ULSMV RMSEASR was consistent across the threshold conditions, except for the
mixed-threshold conditions with small samples.
The uncorrected RMSEA expectedly shared the same asymptotic values as RMSEASR,
but their values under small samples could be very different, especially under the mixed-
threshold condition. However, it was hard to tell whether the uncorrected RMSEA or the
RMSEASR from ULSMV was more appropriate estimates of the asymptotic values. Under the
conditions with mixed thresholds, the uncorrected RMSEA performed better, such as the
conditions with ρ = .3, n = 100, and mixed thresholds, where the defined value of RMSEA was
approximately .171, but the uncorrected RMSEA from ULSMV was .193 and the RMSEASR was
.062. Under some other conditions, the RMSEASR tended to be a better choice, such as the
condition with ρ = .7, n = 100, and asymmetric threshold conditions, where the defined value of
RMSEA was approximately .074, but the uncorrected RMSEA from ULSMV was .147 and the
RMSEASR was .071.
Table 2 presents the RMSEA results for Model 1 with 4-category data and 9 items. The
dependency of the RMSEAPR and RMSEASR from WLSMV and the RMSEAPR from ULSMV on
the threshold values still existed with varying sample sizes, although the dependency tended to
be much smaller compared with the conditions with binary data shown in Table 1.
34
However, Table 2 clearly shows that the RMSEASR from ULSMV was more accurate
than the uncorrected RMSEA under small to moderate sample sizes. For example, under the ρ =
.7 and symmetric-threshold conditions, the RMSEASR from ULSMV went from .069 at n = 100
to .074 at n = 1000, but the uncorrected RMSEA from ULSMV changed from .030 to .073, with
the asymptotic value of .073.
As a summary, only the uncorrected RMSEA and the RMSEASR from ULSMV tended to
be independent of the thresholds with relatively large samples (e.g., n > 500). For binary data,
mixed thresholds, and small sample sizes (e.g., 100 and 200), the uncorrected RMSEA and the
RMSEASR from ULSMV could be both unacceptably inaccurate. When data have four
categories, the RMSEASR from ULSMV was the most accurate because it was stable across the
threshold conditions, and it converged to its asymptotic values faster than the uncorrected
RMSEA.
In addition, Table 1 and 2 both show that the RMSEA from ML was different from the
uncorrected RMSEA, RMSEAPR, and RMSEASR for WLSMV and ULSMV given any sample
size, suggesting that the conventional cutoff values based on continuous data with ML should not
be applied to WLSMV and ULSMV.
CFI and TLI
Table 3 presents the CFI values from Model 1 under the conditions with binary data and
9 items. The dependency of all the CFI values on the thresholds appeared to be ignorable under
large samples, but the uncorrected CFI and CFISR from the ULSMV still appeared to be slightly
more stable across the threshold conditions.
For ULSMV, overall, the CFISR was more accurate compared with the uncorrected CFI,
except for the mixed-threshold conditions. For example, when ρ = .7 and the data distribution
was asymmetric, the CFISR changed from .950 to .972, but the uncorrected CFI increased from
.880 to .965, with the asymptotic value equal to .973. For the mixed-threshold condition, small
samples could result in very inaccurate uncorrected CFI and CFISR. For instance, when ρ = .7 and
the thresholds were mixed, the uncorrected CFI was .897 and the CFISR was .996, with the
corresponding asymptotic value of .973.
When data had 4 categories, as shown in Table 4, the uncorrected RMSEA and
RMSEASR for both WLSMV and ULSMV tended to be stable across the threshold conditions,
35
but the CFIPR was still influenced by the thresholds. The CFISR from ULSMV was clearly closer
to the asymptotic values compared with the uncorrected CFI. For instance, as n increased from
100 to 1000 when ρ = .3 and data were symmetric, the uncorrected CFI decreased from .734 to
.714, while the CFISR decreased from .706 to .711, with the asymptotic value being .712.
Table 5 and 6 present the TLI results from Model 1 under the conditions with 9 items.
The patterns of TLI were similar to those of CFI shown in Table 3 and 4. First, the uncorrected
TLI and the TLISR from ULSMV was the least dependent on the thresholds across all the
conditions. Second, under the binary-data and mixed-threshold conditions, the uncorrected TLI
and TLISR from ULSMV could be very inaccurate when the sample size was 100, 200, or 500.
Third, when data had four categories, the ULSMV TLISR estimates were more accurate under
finite samples compared with the uncorrected TLI under all conditions.
In addition, Table 3 to 6 all showed that the CFI and TLI from ML were different from
the model-fit indexes for WLSMV and ULSMV given any sample size.
36
Figure 1. RMSEA for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories.
37
38
Figure 3. CFI for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories.
39
40
Figure 5. TLI for Model 1, Sample Size 1,000,000, 9 Items, and 2 Ordered Categories.
41
42
Table 1
Means (Standard Deviations) of RMSEA for Model 1, 2-point Scale, and 9 Items
ρ Threshold n ML WLSMV ULSMV
Uncorrected PR SR Uncorrected PR SR
.3 Symmetric 100 .191 (.022) .114 (.034) .127 (.028) .125 (.030) .180 (.032) .113 (.023) .161 (.035)
200 .193 (.015) .120 (.022) .130 (.019) .125 (.020) .176 (.023) .119 (.016) .166 (.025)
500 .196 (.010) .123 (.013) .133 (.012) .125 (.012) .173 (.015) .123 (.010) .169 (.015)
1000 .197 (.007) .125 (.009) .134 (.008) .126 (.009) .173 (.011) .125 (.007) .171 (.011)
1,000,000 .198 .126 .135 .126 .171 .126 .171
Asymmetric 100 .190 (.022) .076 (.038) .085 (.030) .091 (.033) .218 (.036) .067 (.026) .148 (.056)
200 .193 (.015) .086 (.023) .092 (.019) .092 (.021) .201 (.029) .079 (.016) .167 (.035)
500 .196 (.009) .089 (.012) .095 (.011) .091 (.012) .183 (.019) .085 (.009) .169 (.020)
1000 .197 (.007) .091 (.009) .096 (.008) .092 (.009) .178 (.014) .087 (.007) .171 (.014)
1,000,000 .198 .092 .098 .092 .171 .089 .171
Mixed 100 .191 (.021) .026 (.030) .035 (.028) .040 (.032) .193 (.033) .023 (.023) .062 (.060)
200 .193 (.015) .054 (.022) .059 (.019) .061 (.020) .197 (.026) .049 (.016) .134 (.040)
500 .196 (.009) .072 (.011) .075 (.011) .074 (.011) .189 (.019) .066 (.012) .166 (.021)
1000 .197 (.007) .077 (.008) .081 (.007) .078 (.007) .182 (.015) .076 (.011) .172 (.016)
1,000,000 .198 .079 .084 .079 .171 .080 .171
.7 Symmetric 100 .107 (.026) .023 (.029) .058 (.031) .053 (.029) .085 (.034) .052 (.030) .066 (.038)
200 .108 (.016) .033 (.025) .061 (.021) .053 (.019) .080 (.022) .057 (.020) .069 (.025)
500 .108 (.010) .048 (.012) .064 (.011) .055 (.010) .077 (.012) .061 (.010) .072 (.013)
1000 .109 (.007) .052 (.007) .065 (.007) .055 (.007) .075 (.008) .062 (.007) .073 (.009)
1,000,000 .108 .055 .066 .055 .074 .063 .074
Asymmetric 100 .108 (.025) .018 (.028) .044 (.029) .044 (.030) .147 (.042) .037 (.027) .071 (.054)
200 .108 (.016) .019 (.022) .042 (.022) .039 (.021) .115 (.027) .038 (.021) .068 (.038)
500 .108 (.010) .030 (.015) .045 (.012) .040 (.011) .091 (.016) .042 (.011) .071 (.019)
1000 .109 (.007) .036 (.008) .047 (.008) .040 (.007) .083 (.011) .044 (.007) .072 (.013)
1,000,000 .109 .041 .048 .041 .074 .046 .074
Mixed 100 .108 (.026) .002 (.009) .006 (.014) .007 (.017) .125 (.037) .002 (.007) .005 (.020)
200 .108 (.017) .004 (.011) .009 (.015) .010 (.016) .118 (.024) .004 (.009) .011 (.025)
500 .108 (.010) .020 (.014) .029 (.013) .027 (.013) .106 (.017) .023 (.012) .057 (.029)
1000 .108 (.007) .030 (.008) .036 (.008) .034 (.007) .093 (.013) .034 (.008) .073 (.016)
1,000,000 .109 .034 .038 .034 .073 .037 .073
43
Table 2
.3 Symmetric 100 .191 (.022) .174 (.033) .184 (.027) .182 (.031) .159 (.027) .164 (.022) .167 (.025)
200 .193 (.015) .176 (.022) .186 (.018) .180 (.021) .165 (.019) .170 (.016) .168 (.018)
500 .196 (.009) .179 (.013) .189 (.011) .180 (.013) .169 (.011) .174 (.009) .170 (.011)
1000 .197 (.007) .180 (.009) .191 (.008) .180 (.009) .170 (.008) .176 (.007) .171 (.008)
1,000,000 .198 .180 .192 .180 .171 .177 .171
Asymmetric 100 .190 (.022) .171 (.033) .181 (.027) .179 (.031) .159 (.027) .162 (.023) .166 (.026)
200 .193 (.015) .173 (.022) .183 (.019) .177 (.021) .165 (.019) .167 (.016) .169 (.018)
500 .196 (.009) .175 (.013) .186 (.011) .177 (.013) .169 (.011) .171 (.009) .170 (.011)
1000 .197 (.007) .177 (.009) .187 (.008) .177 (.009) .171 (.008) .173 (.007) .171 (.008)
1,000,000 .198 .177 .188 .177 .172 .174 .172
Mixed 100 .191 (.021) .172 (.032) .182 (.027) .180 (.030) .160 (.027) .163 (.022) .167 (.025)
200 .194 (.015) .174 (.022) .184 (.018) .178 (.021) .166 (.019) .168 (.016) .169 (.018)
500 .196 (.009) .175 (.013) .186 (.011) .177 (.013) .170 (.012) .171 (.010) .171 (.012)
1000 .197 (.007) .176 (.009) .187 (.008) .177 (.009) .171 (.008) .173 (.007) .171 (.008)
1,000,000 .198 .176 .188 .176 .171 .174 .171
.7 Symmetric 100 .107 (.027) .046 (.036) .091 (.029) .079 (.027) .030 (.032) .082 (.029) .069 (.025)
200 .108 (.016) .064 (.021) .095 (.018) .080 (.016) .053 (.022) .089 (.017) .072 (.015)
500 .108 (.010) .074 (.011) .098 (.011) .080 (.010) .067 (.010) .093 (.010) .073 (.009)
1000 .109 (.007) .077 (.007) .099 (.008) .080 (.007) .071 (.006) .094 (.007) .074 (.006)
1,000,000 .108 .079 .100 .079 .073 .095 .073
Asymmetric 100 .106 (.026) .045 (.036) .090 (.030) .078 (.027) .032 (.032) .081 (.028) .070 (.025)
200 .107 (.017) .061 (.022) .092 (.019) .077 (.017) .053 (.022) .086 (.018) .071 (.015)
500 .109 (.010) .073 (.011) .097 (.011) .078 (.010) .067 (.010) .091 (.010) .073 (.009)
1000 .108 (.007) .075 (.007) .096 (.008) .078 (.007) .070 (.006) .092 (.007) .073 (.006)
1,000,000 .109 .078 .098 .078 .073 .094 .073
Mixed 100 .107 (.026) .046 (.036) .090 (.030) .078 (.027) .032 (.032) .081 (.028) .069 (.025)
200 .108 (.016) .062 (.022) .093 (.018) .078 (.016) .054 (.021) .087 (.017) .072 (.015)
500 .108 (.010) .073 (.011) .096 (.011) .078 (.010) .067 (.010) .091 (.010) .073 (.010)
1000 .108 (.007) .075 (.007) .097 (.008) .078 (.007) .070 (.007) .092 (.007) .073 (.006)
1,000,000 .109 .078 .098 .078 .074 .093 .074
44
Table 3
Means (Standard Deviations) of CFI for Model 1, 2-point Scale, and 9 Items
.3 Symmetric 100 .533 (.080) .791 (.107) .666 (.127) .754 (.106) .701 (.115) .598 (.160) .718 (.125)
200 .522 (.059) .757 (.081) .642 (.095) .738 (.081) .703 (.090) .588 (.118) .713 (.094)
500 .512 (.038) .738 (.051) .629 (.059) .730 (.051) .710 (.058) .584 (.075) .714 (.059)
1000 .507 (.028) .728 (.038) .620 (.043) .724 (.038) .708 (.043) .578 (.054) .710 (.043)
1,000,000 .502 .721 .615 .721 .711 .578 .711
Asymmetric 100 .537 (.080) .841 (.115) .713 (.150) .791 (.117) .640 (.126) .607 (.223) .728 (.159)
200 .522 (.060) .778 (.100) .662 (.118) .748 (.098) .643 (.112) .567 (.160) .689 (.126)
500 .512 (.039) .749 (.068) .652 (.077) .737 (.068) .682 (.081) .587 (.103) .703 (.084)
1000 .507 (.028) .736 (.049) .645 (.054) .730 (.049) .694 (.059) .589 (.072) .705 (.060)
1,000,000 .504 .723 .621 .723 .713 .583 .713
Mixed 100 .533 (.079) .928 (.102) .846 (.155) .872 (.134) .658 (.124) .849 (.184) .871 (.167)
200 .523 (.057) .824 (.113) .748 (.133) .786 (.115) .665 (.103) .723 (.150) .770 (.125)
500 .511 (.039) .754 (.072) .689 (.081) .741 (.072) .684 (.070) .664 (.093) .722 (.075)
1000 .506 (.028) .729 (.051) .666 (.057) .723 (.051) .693 (.052) .647 (.067) .710 (.054)
1,000,000 .502 .719 .623 .719 .710 .591 .710
.7 Symmetric 100 .879 (.054) .988 (.022) .947 (.046) .968 (.030) .954 (.037) .943 (.053) .967 (.033)
200 .883 (.034) .984 (.017) .949 (.032) .971 (.020) .964 (.021) .948 (.034) .971 (.020)
500 .883 (.021) .977 (.011) .947 (.019) .971 (.011) .969 (.011) .948 (.019) .972 (.011)
1000 .883 (.015) .974 (.008) .947 (.013) .971 (.008) .971 (.008) .948 (.013) .973 (.007)
1,000,000 .884 .971 .945 .971 .973 .948 .973
Asymmetric 100 .879 (.052) .984 (.036) .938 (.068) .961 (.047) .880 (.084) .916 (.101) .950 (.065)
200 .882 (.034) .986 (.024) .948 (.044) .968 (.029) .927 (.045) .941 (.055) .965 (.035)
500 .884 (.021) .981 (.015) .950 (.024) .971 (.015) .956 (.018) .950 (.026) .971 (.016)
1000 .883 (.015) .976 (.011) .950 (.017) .971 (.011) .965 (.011) .951 (.017) .972 (.010)
1,000,000 .884 .971 .946 .971 .973 .948 .973
Mixed 100 .878 (.052) .998 (.010) .990 (.029) .992 (.023) .897 (.069) .995 (.025) .996 (.029)
200 .883 (.035) .997 (.013) .989 (.024) .991 (.020) .927 (.037) .993 (.020) .995 (.016)
500 .883 (.021) .986 (.015) .972 (.021) .978 (.016) .945 (.020) .973 (.021) .979 (.017)
1000 .884 (.015) .977 (.011) .964 (.014) .973 (.011) .957 (.013) .964 (.015) .972 (.012)
1,000,000 .883 .971 .949 .971 .973 .951 .973
45
Table 4
.3 Symmetric 100 .533 (.079) .762 (.089) .611 (.111) .740 (.089) .734 (.106) .530 (.144) .706 (.107)
200 .525 (.059) .749 (.065) .600 (.079) .737 (.065) .726 (.078) .525 (.106) .713 (.078)
500 .511 (.038) .733 (.041) .584 (.049) .729 (.041) .715 (.049) .513 (.066) .710 (.049)
1000 .507 (.028) .730 (.030) .581 (.035) .728 (.030) .714 (.035) .512 (.048) .711 (.035)
1,000,000 .503 .726 .576 .726 .712 .508 .712
Asymmetric 100 .537 (.079) .767 (.088) .619 (.111) .745 (.088) .738 (.104) .542 (.143) .712 (.105)
200 .523 (.059) .746 (.067) .599 (.081) .734 (.067) .722 (.079) .526 (.107) .709 (.080)
500 .510 (.039) .732 (.043) .585 (.050) .728 (.043) .714 (.051) .515 (.068) .708 (.051)
1000 .506 (.028) .728 (.030) .581 (.035) .726 (.030) .712 (.036) .511 (.048) .709 (.036)
1,000,000 .502 .724 .576 .724 .710 .508 .710
Mixed 100 .535 (.078) .764 (.089) .617 (.110) .742 (.089) .735 (.104) .538 (.141) .709 (.105)
200 .523 (.059) .747 (.067) .601 (.081) .735 (.067) .722 (.079) .528 (.106) .710 (.079)
500 .511 (.040) .733 (.044) .587 (.051) .728 (.044) .715 (.052) .518 (.069) .710 (.052)
1000 .506 (.028) .728 (.030) .581 (.035) .725 (.030) .711 (.036) .513 (.048) .709 (.036)
1,000,000 .503 .725 .579 .725 .711 .513 .712
.7 Symmetric 100 .878 (.055) .985 (.019) .935 (.041) .969 (.022) .988 (.018) .935 (.045) .970 (.023)
200 .882 (.035) .979 (.013) .932 (.028) .970 (.014) .983 (.013) .933 (.029) .971 (.014)
500 .883 (.021) .974 (.008) .930 (.017) .970 (.008) .977 (.008) .931 (.017) .972 (.008)
1000 .883 (.015) .973 (.006) .929 (.013) .971 (.006) .975 (.005) .931 (.013) .972 (.006)
1,000,000 .884 .971 .927 .971 .973 .930 .973
Asymmetric 100 .880 (.055) .984 (.019) .936 (.041) .969 (.023) .987 (.019) .935 (.044) .969 (.023)
200 .883 (.035) .980 (.013) .935 (.028) .971 (.014) .982 (.013) .935 (.029) .972 (.014)
500 .882 (.021) .974 (.008) .929 (.018) .970 (.009) .976 (.008) .931 (.018) .972 (.008)
1000 .884 (.014) .973 (.006) .930 (.012) .971 (.006) .975 (.005) .932 (.012) .973 (.005)
1,000,000 .884 .971 .928 .971 .973 .930 .973
Mixed 100 .879 (.055) .985 (.019) .937 (.040) .969 (.022) .988 (.018) .936 (.043) .970 (.022)
200 .882 (.035) .980 (.013) .935 (.027) .971 (.014) .982 (.013) .935 (.028) .972 (.014)
500 .883 (.021) .974 (.008) .931 (.017) .970 (.009) .976 (.008) .933 (.018) .972 (.008)
1000 .884 (.015) .973 (.006) .930 (.013) .971 (.006) .975 (.006) .932 (.013) .973 (.006)
1,000,000 .883 .970 .928 .970 .972 .930 .972
46
Table 5
Means (Standard Deviations) of TLI for Model 1, 2-point Scale, and 9 Items
.3 Symmetric 100 .377 (.106) .722 (.142) .555 (.170) .672 (.141) .601 (.153) .464 (.211) .624 (.167)
200 .363 (.079) .676 (.108) .523 (.126) .650 (.108) .604 (.120) .450 (.158) .617 (.125)
500 .350 (.051) .650 (.069) .505 (.079) .640 (.068) .613 (.078) .445 (.099) .618 (.079)
1000 .343 (.038) .637 (.050) .493 (.057) .632 (.050) .611 (.057) .437 (.072) .614 (.058)
1,000,000 .337 .628 .487 .628 .615 .437 .615
Asymmetric 100 .383 (.106) .787 (.154) .617 (.199) .721 (.156) .520 (.168) .485 (.278) .637 (.211)
200 .362 (.080) .703 (.134) .549 (.157) .664 (.131) .524 (.149) .424 (.209) .586 (.167)
500 .349 (.052) .665 (.091) .536 (.103) .649 (.091) .576 (.108) .449 (.137) .603 (.112)
1000 .343 (.038) .648 (.065) .527 (.072) .640 (.065) .592 (.078) .453 (.096) .607 (.080)
1,000,000 .339 .631 .495 .631 .617 .444 .617
Mixed 100 .378 (.106) .901 (.142) .793 (.209) .830 (.175) .543 (.166) .791 (.252) .831 (.210)
200 .364 (.076) .765 (.151) .664 (.178) .715 (.154) .553 (.138) .632 (.199) .694 (.166)
500 .348 (.052) .673 (.096) .586 (.108) .655 (.096) .578 (.093) .552 (.124) .630 (.100)
1000 .342 (.037) .639 (.068) .555 (.076) .630 (.068) .590 (.069) .529 (.089) .614 (.072)
1,000,000 .336 .626 .497 .626 .614 .454 .614
.7 Symmetric 100 .838 (.072) .983 (.030) .929 (.062) .958 (.040) .939 (.050) .924 (.071) .957 (.043)
200 .844 (.045) .979 (.023) .931 (.043) .961 (.026) .952 (.029) .930 (.045) .962 (.026)
500 .844 (.028) .969 (.015) .929 (.025) .961 (.015) .959 (.015) .930 (.026) .963 (.015)
1000 .845 (.020) .966 (.010) .929 (.017) .961 (.010) .961 (.010) .931 (.017) .963 (.010)
1,000,000 .845 .961 .927 .961 .963 .930 .963
Asymmetric 100 .838 (.069) .978 (.048) .917 (.090) .947 (.062) .840 (.112) .888 (.135) .933 (.086)
200 .843 (.045) .981 (.031) .931 (.058) .958 (.039) .902 (.060) .921 (.073) .953 (.047)
500 .845 (.028) .974 (.020) .934 (.032) .961 (.021) .941 (.024) .933 (.034) .961 (.021)
1000 .844 (.020) .968 (.014) .933 (.022) .961 (.014) .953 (.015) .934 (.023) .963 (.014)
1,000,000 .845 .961 .928 .961 .963 .931 .963
Mixed 100 .837 (.070) .997 (.014) .986 (.039) .989 (.031) .862 (.091) .992 (.040) .994 (.033)
200 .844 (.047) .995 (.017) .985 (.032) .988 (.026) .903 (.049) .991 (.027) .993 (.021)
500 .844 (.029) .982 (.019) .963 (.028) .971 (.022) .926 (.026) .964 (.028) .971 (.023)
1000 .845 (.020) .970 (.015) .952 (.019) .964 (.015) .943 (.017) .951 (.020) .962 (.016)
1,000,000 .844 .961 .933 .961 .963 .935 .963
47
Table 6
.3 Symmetric 100 .377 (.106) .683 (.119) .481 (.148) .654 (.119) .645 (.141) .375 (.188) .608 (.143)
200 .367 (.079) .665 (.087) .466 (.105) .650 (.087) .635 (.103) .368 (.140) .617 (.104)
500 .348 (.051) .644 (.055) .446 (.065) .638 (.055) .620 (.065) .351 (.088) .613 (.065)
1000 .343 (.037) .640 (.040) .442 (.046) .637 (.040) .619 (.047) .349 (.064) .615 (.047)
1,000,000 .338 .635 .435 .635 .616 .344 .616
Asymmetric 100 .382 (.106) .690 (.118) .493 (.148) .660 (.118) .651 (.138) .391 (.188) .617 (.140)
200 .364 (.078) .661 (.089) .466 (.108) .646 (.089) .629 (.105) .368 (.141) .612 (.106)
500 .347 (.052) .643 (.058) .447 (.067) .637 (.058) .618 (.068) .353 (.090) .612 (.068)
1000 .341 (.037) .638 (.041) .441 (.047) .635 (.041) .616 (.048) .348 (.065) .612 (.048)
1,000,000 .336 .632 .435 .632 .613 .345 .613
Mixed 100 .379 (.104) .685 (.118) .489 (.146) .656 (.119) .646 (.139) .386 (.184) .612 (.141)
200 .364 (.079) .662 (.089) .469 (.107) .647 (.089) .630 (.105) .371 (.141) .613 (.106)
500 .348 (.053) .644 (.058) .450 (.068) .638 (.058) .619 (.069) .358 (.092) .613 (.069)
1000 .341 (.037) .637 (.040) .442 (.047) .634 (.040) .615 (.047) .351 (.064) .612 (.048)
1,000,000 .337 .634 .438 .634 .615 .350 .615
.7 Symmetric 100 .838 (.073) .980 (.025) .914 (.055) .959 (.029) .984 (.024) .913 (.060) .960 (.030)
200 .843 (.047) .973 (.018) .910 (.037) .960 (.019) .977 (.018) .911 (.039) .962 (.018)
500 .844 (.028) .966 (.011) .906 (.023) .961 (.011) .969 (.010) .908 (.023) .963 (.010)
1000 .845 (.020) .963 (.008) .905 (.017) .961 (.008) .966 (.007) .908 (.017) .963 (.007)
1,000,000 .846 .961 .903 .961 .964 .907 .964
Asymmetric 100 .841 (.073) .979 (.026) .915 (.055) .959 (.030) .983 (.025) .914 (.059) .959 (.031)
200 .845 (.046) .974 (.018) .913 (.037) .961 (.019) .977 (.018) .914 (.038) .963 (.018)
500 .843 (.028) .965 (.011) .906 (.023) .960 (.011) .968 (.011) .908 (.024) .962 (.011)
1000 .845 (.019) .964 (.008) .906 (.016) .961 (.008) .966 (.007) .909 (.016) .964 (.007)
1,000,000 .845 .961 .904 .961 .964 .907 .964
Mixed 100 .839 (.073) .980 (.025) .916 (.053) .959 (.029) .984 (.024) .915 (.057) .960 (.030)
200 .843 (.046) .973 (.018) .913 (.036) .961 (.018) .976 (.018) .914 (.037) .962 (.018)
500 .844 (.028) .966 (.011) .908 (.023) .961 (.011) .968 (.011) .910 (.024) .963 (.011)
1000 .845 (.020) .964 (.008) .907 (.017) .961 (.008) .966 (.007) .910 (.017) .963 (.008)
1,000,000 .844 .960 .904 .960 .963 .907 .963
48
CHAPTER 5
DISCUSSION AND CONCLUSIONS
The present study investigates the chi-square-based model fit indexes, including RMSEA,
CFI and TLI, when WLSMV and ULSMV are applied to ordered categorical data in SEM. The
current practice to evaluate global model-data fit is to apply the PR fit indexes which are
implemented in Mplus and calculated by replacing the uncorrected chi-square statistic with the
mean- and variance-adjusted chi-square. When a hypothesized model is correctly specified,
RMSEAPR, CFIPR, and TLIPR converge to their defined values and do not depend on other factors
such as sample sizes, thresholds, numbers of ordered categories, and degrees of nonnormality for
the underlying continuous variables (DiStefano & Morgan, 2014). However, under misspecified
models, the PR fit indexes from WLSMV have been evidenced to be inadvisable because they
depend on different design factors (Sass et al., 2014; Yang & Xia, 2015; Yu, 2002). The fit
indexes from ULSMV haven’t been investigated systematically. It is therefore important to
investigate and improve the model-data fit evaluation techniques for practitioners. The
alternatives to the PR fit indexes are the SR fit indexes. Based on Brosseau-Liard et al. (2012)
and Brosseau-Liard and Savalei (2014), under misspecified models, the PR fit indexes do not
converge to the original definitions of RMSEA, CFI, and TLI, but the SR fit indexes do. Because
the SR fit indexes haven’t been extended to WLSMV and ULSMV, it is worthwhile to
investigate them and compare them with their PR counterparts.
One focus of the current study is on the dependency of the PR and SR fit indexes from
WLSMV and ULSMV on the thresholds. When the misspecification lies in the factor structures
and not in the thresholds, a good model-fit index should ideally reflect the degree of
misspecification, rather than being confounded with the threshold values. This dissertation thus
first investigates the PR and SR model-fit indexes from WLSMV and ULSMV with an
extremely large sample size (i.e., 1,000,000) to examine the dependency of these indexes on the
thresholds. The results show that only the SR fit indexes from ULSMV are stable across the
threshold conditions, holding the other design factors constant. Second, a simulation study with
different finite sample sizes, numbers of categories, numbers of items, threshold conditions, and
data generation models is implemented to further evaluate the performance of these indexes at
49
the sample level. In addition, because the uncorrected RMSEA, CFI, and TLI without robust
correction share the same asymptotic values as the SR fit indexes, they are compared with the PR
fit indexes and SR fit indexes regarding their accuracy under small to moderate sample sizes.
The results find that the SR fit indexes from ULSMV are stable across the threshold conditions
under finite samples and are the most accurate estimates of their asymptotic values in general.
I discuss below the implications and future research that is needed for the model-fit
indexes when ordered categorical data are analyzed in SEM.
The Dependency of Model Fit Indexes on Thresholds
The PR model-fit indexes have been used in almost every application where WLSMV is
implemented for ordered categorical data (e.g., Ayer et al., 2009; Eaton et al., 2013; Keyes et al.,
2013). However, recent literature suggests to use ULSMV unless it results in nonconvergent
solutions (Forero et al., 2009; Li, 2014; Yang-Wallentin et al., 2010). I’ve found 16 studies in
substantive areas (e.g., Currier & Holland, 2014; De Beer, Pienaar, Rothmann, 2014; Stander,
2014) that used ULSMV along with the PR fit indexes to evaluate model data fit. However, the
present study shows that the values of the PR fit indexes from both WLSMV and ULSMV are
influenced by the thresholds, especially for binary data, and large degrees of misspecification
(i.e., the conditions with ρ = .3). Because an extremely large sample size (i.e., 1,000,000) is used
in the present study, the dependency of the PR fit indexes from WLSMV and ULSMV is a
property of the indexes and estimators rather than the results of sample fluctuations.
As mentioned above, the dependency of fit indexes on data distribution is not a desirable
property. This is because when the true model and the hypothesized model both remain
unchanged, they indicate different degrees of model-data fit, depending on the thresholds. For
example, a model with an RMSEAPR of .03 is considered to show acceptable fit based on the
conventional cutoff for RMSEA. It is possible that the model explains reasonably well the
relationships among the variables. However, it is also possible that the hypothesized model is
severely misspecified, but data have asymmetric thresholds, making the RMSEAPR much less
sensitive to detect the model misspecification. If a model fit index is dependent on the thresholds
asymptotically, then the cutoff values should be threshold specific.
The SR fit indexes (Brosseau-Liard et al., 2012; Brosseau-Liard & Savalei, 2014; Li &
Bentler, 2006) perform better than the PR fit indexes because the former converge to the
50
definitions of RMSEA, CFI, and TLI in (8), (10), and (11), but the latter converge to the
distorted values shown in (17), (18), and (19). For WLSMV, the SR fit indexes still depend on
the threshold values, especially under binary data with a large degree of model misspecification.
For ULSMV, however, the SR fit indexes are stable across all the threshold conditions, given the
other design factors. From this point of view, the SR fit indexes from ULSMV are the most
appropriate for model-data evaluation compared with the PR alternatives, and the PR and SR fit
indexes from WLSMV.
The reasons for the dependency of fit indexes on the thresholds are twofold. First, the
only difference between the WLSMV SR fit indexes and the ULSMV SR fit indexes is the
weight matrix contained in the corresponding fit functions. The ULSMV SR fit indexes are
independent of threshold values asymptotically but those from WLSMV are, evidencing that the
weight matrix in the WLSMV fit function makes its SR fit indexes dependent on the thresholds.
Second, the asymptotic values of SR and PR fit indexes from ULSMV differ only by the scale
and shift factors as shown in (8), (10), (11), (17), (18), and (19). Therefore, the scale and shift
factors in the robust correction also make the PR fit indexes depend on the thresholds.
The Accuracy of Model Fit Indexes
In the first part of this simulation study, estimates from an extremely large sample are
used to approximate the population values. The second part of the simulation study varies the
sample sizes from 100 to 1000 to investigate how close the model fit indexes under finite
samples are to their defined population values approximated by using the extremely large
samples.
As expected, the SR fit indexes converge to the definitions of RMSEA, CFI and TLI as
shown in (8), (10), and (11), respectively. In addition, the SR fit indexes from ULSMV are stable
across the threshold conditions, consistent with the results under the sample size of 1,000,000.
The PR model fit indexes converge to values that are different from the definitions of RMSEA,
CFI and TLI, and are influenced by thresholds.
Because the uncorrected model-fit indexes share the same asymptotic values as the SR fit
indexes, the present study compares their accuracy under varying conditions. Overall, the SR fit
indexes from ULSMV under finite samples are accurate estimates of their asymptotic values,
except for the condition where the items have mixed thresholds, data are binary, and sample
51
sizes are 100 or 200. The uncorrected model fit indexes tend to be less accurate than the SR fit
indexes, especially under the 4-categories conditions.
My suggestion is to apply the SR model fit indexes with ULSMV. However, as
mentioned above, the results under the conditions with binary data, different thresholds across
items, and small samples can give very inaccurate results when the ULSMV SR fit indexes are
calculated. Therefore, for substantive research, it would be wise to check the estimated
thresholds when binary data are analyzed. If the estimated thresholds tend to be very different
across items, a sample size of at least 500 is needed to obtain accurate estimates of the model fit
indexes.
The Use of Conventional Cutoff Values
Although lots of arguments exist that are against the generalizability of the conventional
cutoff values developed under continuous data with ML (e.g., Marsh et al., 2004), the current
practice is still to apply them to make decisions about model-data fit when WLSMV or ULSMV
is used (e.g., Currier & Holland, 2014; Eaton et al., 2013; Keyes et al., 2013). The current study
includes the model-fit indexes based on the ML estimator before continuous data are categorized.
Results show that the RMSEA, CFI, and TLI from ML can have very different values from the
PR and SR fit indexes based on WLSMV and ULSMV. The results further support of the
arguments that the conventional cutoff values should not be applied universally (e.g., Marsh et
al., 2004; Yu, 2002).
For the majority of the conditions selected in the present study, the RMSEA from ML has
higher values compared with the RMSEAPR and RMSEASR from WLSMV and ULSMV, the CFI
and TLI from ML have lower values compared with the corresponding values from WLSMV and
ULSMV. Therefore, using the PR fit indexes from WLSMV with the conventional rules more
likely leads to the acceptance of a model that is highly misspecified. In the long run, applying the
conventional rules to WLSMV and ULSMV will lead to the accumulation of largely
misspecified models in publications. In conclusion, the model fit indexes from WLSMV are not
suggested because they depend on the thresholds asymptotically and the cutoff values should be
threshold specific. If ULSMV is applied, the conventional cutoff values may not be applicable
and should be re-evaluated. However, because the SR fit indexes from ULSMV are not available
52
in the software, and the performance of its fit indexes has not been systematically investigated,
what cutoff values are appropriate need further evaluation.
Future Research
Several limitations exist in the present dissertation. First, an extremely large sample size
is used to approximate the asymptotic values of the model fit indexes. The results can be more
accurate if the influence of thresholds on model fit indexes is investigated by their exact
formulas. The conclusions from the current results will also be more convincing if the
dependency of the indexes on the thresholds can be proved mathematically.
Second, the current simulation study generates data using two simple CFA models, which
limits the generalizability of our conclusions. The SEM methods include more complex models
that are widely applied, such as longitudinal models, measurement invariance models, mixture
models. Based on the findings from the current study, the SR fit indexes from ULSMV are
suggested. Future research should investigate their behaviors under varying simulation designs.
Third, as mentioned above, if the conventional cutoffs shouldn’t be used when the SR fit
indexes from ULSMV are applied, then what appropriate cutoff values can be used? This is an
important question that needs to be addressed, because applied researchers often rely on cutoff
values to evaluate model-data fit.
Conclusions
The simulation study aims to address the three research questions brought up in the first
two chapters.
the thresholds for categorization under extremely large samples? Expectedly, only the SR fit
indexes for ULSMV have asymptotic values not dependent on the thresholds.
approximations of the definitions of RMSEA, CFI and TLI compared with the PR fit indexes and
the uncorrected model-fit indexes? The answer is yes. However, if thresholds vary greatly across
items and data are binary, a sample size greater than 500 observations is needed to obtain
accurate estimates of model fit indexes. In addition, the results support the expectation that the
53
SR fit indexes converge to the original definitions, but the PR fit indexes converge to distorted
values.
Third, if continuous data before categorization are analyzed by ML, to what degree are
the RMSEA, CFI, and TLI different from the corresponding model fit indexes based on WLSMV
and ULSMV after categorization? The results show that the model fit indexes from ML have
different values from the PR and SR fit indexes based on WLSMV and ULSMV. In general, the
RMSEA from ML is larger than the RMSEAPR and RMSEASR from WLSMV and ULSMV, and
the CFI and TLI from ML are smaller than those from WLSMV and ULSMV.
As a summary, the results from this dissertation suggest the use of the SR model fit
indexes with the ULSMV estimator, because of the following reasons: (1) They are not
dependent on the thresholds asymptotically, given that the model misspecification lies in the
factor structures and not in the thresholds; (2) They converge to the defined values of RMSEA,
CFI, and TLI, and are the most accurate measures of their defined values under finite sample
sizes. However, a sample size larger than 500 are needed if items have different thresholds and
data are binary. Furthermore, the conventional cutoff values developed under continuous data
with ML should not be applied in WLSMV and ULSMV estimators.
54
APPENDIX A
RESULTS FROM THE SIMULATION AT THE POPULATION LEVEL
55
56
Figure 9. RMSEA for Model 2, Sample Size 1,000,000, and 2 Ordered Categories.
57
Figure 10. RMSEA for Model 2, Sample Size 1,000,000, and 4 Ordered Categories.
58
59
60
Figure 13. CFI for Model 2, Sample Size 1,000,000, and 2 Ordered Categories.
61
Figure 14. CFI for Model 2, Sample Size 1,000,000, and 4 Ordered Categories.
62
63
64
Figure 17. TLI for Model 2, Sample Size 1,000,000, and 2 Ordered Categories.
65
Figure 18. TLI for Model 2, Sample Size 1,000,000, and 4 Ordered Categories.
66
APPENDIX B
RESULTS FROM THE SIMULATION AT THE SAMPLE LEVEL
67
Table 7
.3 Symmetric 100 .158 (.012) .116 (.022) .108 (.015) .126 (.019) .179 (.022) .096 (.012) .162 (.023)
200 .156 (.008) .118 (.014) .113 (.010) .123 (.013) .173 (.016) .103 (.009) .163 (.017)
500 .157 (.005) .119 (.008) .118 (.006) .121 (.008) .169 (.010) .110 (.006) .165 (.010)
1000 .158 (.004) .121 (.006) .120 (.004) .122 (.006) .168 (.007) .113 (.004) .166 (.007)
1,000,000 .158 .121 .123 .121 .166 .116 .166
Asymmetric 100 .158 (.012) .082 (.025) .072 (.014) .096 (.020) .222 (.025) .057 (.013) .159 (.033)
200 .157 (.008) .085 (.015) .077 (.010) .091 (.014) .198 (.021) .067 (.009) .166 (.023)
500 .157 (.005) .087 (.008) .083 (.006) .090 (.008) .180 (.013) .075 (.006) .166 (.014)
1000 .158 (.004) .088 (.006) .086 (.005) .089 (.006) .173 (.010) .079 (.004) .166 (.010)
1,000,000 .158 .088 .090 .088 .166 .083 .166
Mixed 100 .158 (.012) .041 (.026) .038 (.014) .058 (.021) .199 (.020) .021 (.013) .074 (.047)
200 .157 (.009) .063 (.012) .054 (.009) .069 (.011) .195 (.016) .037 (.007) .136 (.023)
500 .157 (.005) .073 (.006) .071 (.006) .075 (.006) .185 (.011) .050 (.009) .162 (.012)
1000 .158 (.004) .077 (.004) .077 (.004) .078 (.004) .178 (.009) .067 (.009) .168 (.009)
1,000,000 .158 .079 .082 .079 .167 .077 .167
.7 Symmetric 100 .096 (.012) .018 (.022) .053 (.014) .057 (.016) .086 (.020) .047 (.013) .069 (.021)
200 .092 (.008) .033 (.017) .055 (.010) .054 (.010) .079 (.013) .051 (.009) .070 (.013)
500 .092 (.005) .047 (.007) .059 (.006) .054 (.006) .075 (.007) .056 (.005) .071 (.007)
1000 .092 (.003) .050 (.004) .061 (.004) .053 (.004) .073 (.005) .058 (.004) .071 (.005)
1,000,000 .092 .053 .063 .053 .071 .060 .071
Asymmetric 100 .096 (.013) .013 (.020) .039 (.013) .049 (.017) .150 (.028) .032 (.013) .077 (.033)
200 .092 (.009) .017 (.017) .038 (.010) .042 (.011) .114 (.017) .034 (.010) .072 (.021)
500 .092 (.005) .030 (.009) .041 (.006) .040 (.006) .090 (.010) .038 (.005) .071 (.011)
1000 .092 (.004) .035 (.005) .043 (.004) .039 (.004) .081 (.007) .040 (.004) .071 (.007)
1,000,000 .092 .040 .046 .040 .072 .044 .072
Mixed 100 .095 (.013) .001 (.003) .003 (.007) .005 (.012) .128 (.020) .001 (.002) .001 (.006)
200 .093 (.008) .001 (.005) .007 (.009) .009 (.012) .116 (.014) .001 (.003) .002 (.010)
500 .092 (.005) .022 (.008) .029 (.006) .030 (.006) .104 (.008) .017 (.006) .057 (.015)
1000 .092 (.003) .031 (.004) .035 (.004) .034 (.004) .091 (.007) .030 (.006) .072 (.008)
1,000,000 .092 .034 .038 .034 .071 .035 .071
68
Table 8
.3 Symmetric 100 .157 (.012) .171 (.024) .151 (.015) .180 (.022) .156 (.021) .136 (.013) .164 (.019)
200 .157 (.009) .171 (.015) .156 (.010) .175 (.015) .161 (.014) .144 (.009) .165 (.013)
500 .157 (.005) .172 (.009) .161 (.006) .174 (.009) .164 (.008) .150 (.005) .166 (.008)
1000 .158 (.004) .172 (.006) .164 (.004) .173 (.006) .165 (.006) .153 (.004) .166 (.006)
1,000,000 .158 .173 .166 .173 .167 .156 .167
Asymmetric 100 .158 (.012) .169 (.024) .150 (.015) .178 (.022) .156 (.021) .135 (.013) .164 (.019)
200 .157 (.009) .169 (.015) .154 (.010) .173 (.015) .162 (.014) .142 (.009) .165 (.013)
500 .157 (.005) .169 (.010) .159 (.006) .170 (.009) .164 (.009) .148 (.005) .166 (.009)
1000 .158 (.004) .169 (.007) .161 (.004) .170 (.007) .166 (.006) .150 (.004) .166 (.006)
1,000,000 .158 .170 .163 .170 .167 .153 .167
Mixed 100 .157 (.012) .169 (.024) .150 (.015) .178 (.022) .156 (.021) .135 (.013) .164 (.019)
200 .157 (.008) .169 (.016) .155 (.010) .173 (.015) .162 (.014) .142 (.009) .165 (.014)
500 .157 (.005) .168 (.009) .159 (.006) .170 (.009) .164 (.009) .148 (.006) .165 (.009)
1000 .158 (.004) .169 (.007) .161 (.004) .170 (.007) .166 (.006) .151 (.004) .166 (.006)
1,000,000 .158 .170 .164 .170 .167 .153 .167
.7 Symmetric 100 .096 (.013) .044 (.027) .078 (.014) .080 (.015) .024 (.025) .071 (.013) .070 (.014)
200 .092 (.009) .062 (.013) .083 (.010) .078 (.010) .051 (.015) .077 (.009) .070 (.009)
500 .092 (.005) .071 (.007) .088 (.006) .077 (.006) .064 (.006) .084 (.006) .071 (.006)
1000 .092 (.004) .074 (.004) .091 (.004) .077 (.004) .068 (.004) .087 (.004) .071 (.004)
1,000,000 .092 .077 .094 .077 .071 .090 .071
Asymmetric 100 .096 (.013) .044 (.026) .077 (.014) .080 (.015) .026 (.026) .070 (.013) .070 (.014)
200 .093 (.008) .062 (.013) .082 (.009) .077 (.010) .053 (.015) .076 (.009) .071 (.009)
500 .092 (.005) .070 (.007) .087 (.006) .076 (.006) .065 (.007) .082 (.006) .071 (.006)
1000 .092 (.003) .073 (.004) .089 (.004) .075 (.004) .068 (.004) .085 (.004) .071 (.004)
1,000,000 .092 .075 .092 .075 .071 .088 .071
Mixed 100 .096 (.012) .045 (.027) .078 (.013) .080 (.015) .027 (.026) .070 (.013) .071 (.014)
200 .092 (.008) .061 (.013) .081 (.010) .076 (.010) .052 (.014) .076 (.009) .071 (.009)
500 .092 (.005) .070 (.007) .086 (.006) .076 (.006) .065 (.007) .082 (.006) .071 (.006)
1000 .092 (.004) .072 (.004) .089 (.004) .075 (.004) .068 (.004) .085 (.004) .071 (.004)
1,000,000 .092 .075 .092 .075 .071 .088 .071
69
Table 9
.3 Symmetric 100 .145 (.023) .031 (.033) .077 (.031) .067 (.028) .090 (.036) .069 (.030) .082 (.036)
200 .146 (.016) .051 (.024) .083 (.020) .070 (.017) .093 (.023) .077 (.018) .089 (.023)
500 .147 (.010) .065 (.011) .087 (.012) .071 (.010) .093 (.013) .082 (.011) .092 (.013)
1000 .147 (.007) .068 (.007) .087 (.008) .071 (.007) .093 (.009) .083 (.007) .092 (.009)
1,000,000 .148 .072 .089 .072 .093 .084 .093
Asymmetric 100 .146 (.023) .017 (.027) .051 (.030) .049 (.029) .139 (.044) .042 (.028) .077 (.053)
200 .146 (.016) .027 (.025) .056 (.022) .050 (.020) .121 (.030) .050 (.021) .086 (.038)
500 .147 (.010) .043 (.013) .061 (.012) .052 (.010) .105 (.018) .056 (.011) .091 (.019)
1000 .147 (.007) .049 (.008) .063 (.008) .052 (.007) .099 (.012) .058 (.007) .093 (.013)
1,000,000 .148 .052 .064 .052 .093 .060 .093
Mixed 100 .146 (.023) .002 (.009) .006 (.014) .008 (.017) .124 (.041) .003 (.009) .008 (.025)
200 .146 (.016) .004 (.011) .010 (.015) .011 (.017) .118 (.028) .005 (.010) .014 (.029)
500 .147 (.010) .023 (.015) .032 (.013) .031 (.012) .114 (.017) .018 (.012) .054 (.033)
1000 .147 (.007) .037 (.007) .043 (.008) .043 (.007) .108 (.013) .032 (.012) .085 (.017)
1,000,000 .148 .043 .048 .043 .092 .046 .092
70
Table 10
.3 Symmetric 100 .144 (.023) .070 (.036) .124 (.028) .102 (.024) .048 (.036) .113 (.026) .089 (.022)
200 .147 (.016) .090 (.019) .131 (.019) .104 (.016) .075 (.019) .121 (.017) .092 (.015)
500 .147 (.010) .098 (.010) .134 (.011) .103 (.010) .087 (.010) .125 (.010) .093 (.009)
1000 .148 (.007) .101 (.007) .135 (.008) .103 (.007) .090 (.006) .126 (.007) .093 (.006)
1,000,000 .148 .103 .136 .103 .093 .128 .093
Asymmetric 100 .144 (.024) .069 (.036) .122 (.028) .101 (.024) .049 (.037) .111 (.026) .089 (.023)
200 .146 (.016) .088 (.019) .127 (.018) .101 (.016) .075 (.019) .118 (.017) .092 (.015)
500 .147 (.010) .096 (.010) .131 (.011) .101 (.010) .087 (.010) .122 (.010) .092 (.009)
1000 .147 (.007) .099 (.007) .132 (.008) .101 (.007) .090 (.007) .123 (.007) .093 (.006)
1,000,000 .148 .101 .133 .101 .093 .125 .093
Mixed 100 .145 (.023) .071 (.036) .123 (.027) .101 (.024) .051 (.036) .111 (.026) .090 (.022)
200 .147 (.016) .088 (.019) .127 (.019) .101 (.016) .076 (.019) .118 (.017) .092 (.015)
500 .147 (.010) .096 (.011) .130 (.012) .101 (.010) .087 (.010) .122 (.011) .093 (.009)
1000 .148 (.007) .099 (.007) .131 (.008) .101 (.007) .090 (.007) .123 (.007) .093 (.007)
1,000,000 .148 .101 .133 .101 .093 .124 .093
71
Table 11
.3 Symmetric 100 .086 (.012) .020 (.023) .053 (.015) .057 (.016) .094 (.020) .045 (.015) .070 (.024)
200 .082 (.008) .038 (.016) .056 (.010) .056 (.010) .085 (.013) .051 (.009) .073 (.014)
500 .081 (.004) .050 (.007) .059 (.006) .056 (.006) .079 (.008) .055 (.005) .074 (.008)
1000 .081 (.003) .053 (.004) .060 (.004) .056 (.004) .077 (.005) .056 (.004) .074 (.005)
1,000,000 .081 .056 .062 .056 .075 .058 .075
Asymmetric 100 .086 (.012) .012 (.019) .037 (.014) .047 (.018) .159 (.023) .028 (.014) .072 (.036)
200 .082 (.008) .019 (.017) .038 (.010) .043 (.011) .124 (.017) .033 (.010) .076 (.023)
500 .081 (.004) .033 (.008) .041 (.006) .041 (.006) .096 (.010) .037 (.005) .075 (.012)
1000 .081 (.003) .037 (.005) .042 (.004) .041 (.004) .086 (.007) .039 (.004) .075 (.008)
1,000,000 .081 .041 .045 .041 .074 .041 .074
Mixed 100 .086 (.012) .001 (.006) .006 (.010) .010 (.016) .151 (.019) .001 (.004) .005 (.016)
200 .082 (.008) .003 (.008) .010 (.010) .014 (.014) .131 (.014) .004 (.007) .016 (.025)
500 .081 (.004) .021 (.009) .026 (.006) .029 (.007) .107 (.009) .014 (.005) .054 (.019)
1000 .081 (.003) .029 (.004) .032 (.004) .032 (.004) .094 (.007) .021 (.006) .069 (.009)
1,000,000 .081 .033 .035 .033 .074 .035 .074
72
Table 12
.3 Symmetric 100 .086 (.012) .051 (.027) .080 (.014) .084 (.016) .035 (.027) .069 (.013) .074 (.015)
200 .082 (.008) .067 (.012) .083 (.010) .081 (.010) .059 (.013) .075 (.009) .074 (.010)
500 .081 (.004) .075 (.006) .087 (.006) .080 (.006) .069 (.006) .079 (.005) .074 (.006)
1000 .081 (.003) .077 (.004) .089 (.004) .080 (.004) .072 (.004) .081 (.004) .075 (.004)
1,000,000 .081 .080 .091 .080 .075 .083 .075
Asymmetric 100 .087 (.012) .051 (.026) .079 (.014) .084 (.015) .038 (.027) .069 (.013) .074 (.015)
200 .082 (.008) .066 (.013) .082 (.009) .080 (.010) .059 (.013) .073 (.009) .074 (.010)
500 .081 (.004) .074 (.006) .086 (.006) .079 (.006) .069 (.006) .078 (.005) .075 (.006)
1000 .081 (.003) .076 (.004) .087 (.004) .079 (.004) .072 (.004) .080 (.004) .075 (.004)
1,000,000 .081 .078 .089 .078 .075 .082 .075
Mixed 100 .086 (.012) .050 (.027) .078 (.014) .082 (.016) .037 (.028) .068 (.013) .074 (.015)
200 .082 (.007) .066 (.012) .082 (.009) .080 (.010) .060 (.013) .074 (.008) .074 (.009)
500 .082 (.004) .074 (.006) .085 (.006) .079 (.006) .069 (.006) .078 (.005) .075 (.006)
1000 .081 (.003) .076 (.004) .087 (.004) .079 (.004) .072 (.004) .080 (.004) .075 (.004)
1,000,000 .081 .078 .088 .078 .074 .081 .074
73
Table 13
.3 Symmetric 100 .497 (.060) .787 (.080) .647 (.096) .749 (.081) .692 (.093) .575 (.124) .712 (.097)
200 .495 (.045) .762 (.062) .625 (.072) .741 (.062) .703 (.071) .564 (.092) .714 (.073)
500 .490 (.030) .745 (.039) .606 (.045) .737 (.039) .710 (.045) .550 (.058) .715 (.046)
1000 .489 (.021) .739 (.028) .596 (.032) .735 (.028) .712 (.032) .542 (.041) .715 (.033)
1,000,000 .487 .734 .588 .734 .717 .535 .717
Asymmetric 100 .498 (.060) .829 (.090) .684 (.107) .775 (.088) .599 (.113) .554 (.164) .694 (.124)
200 .496 (.044) .786 (.073) .650 (.086) .754 (.072) .638 (.094) .544 (.125) .691 (.098)
500 .490 (.029) .758 (.047) .634 (.053) .745 (.047) .681 (.059) .558 (.074) .705 (.060)
1000 .488 (.021) .750 (.034) .630 (.037) .743 (.034) .699 (.042) .563 (.051) .711 (.042)
1,000,000 .487 .740 .624 .740 .716 .564 .716
Mixed 100 .496 (.060) .893 (.094) .788 (.119) .822 (.103) .614 (.103) .833 (.146) .863 (.121)
200 .493 (.045) .799 (.078) .721 (.089) .761 (.079) .650 (.078) .732 (.100) .770 (.088)
500 .491 (.029) .749 (.048) .670 (.055) .735 (.048) .680 (.040) .685 (.060) .726 (.052)
1000 .489 (.021) .733 (.033) .652 (.037) .726 (.033) .694 (.035) .651 (.044) .715 (.035)
1,000,000 .487 .723 .643 .723 .715 .632 .715
.7 Symmetric 100 .831 (.044) .992 (.014) .939 (.033) .966 (.021) .953 (.028) .939 (.036) .967 (.022)
200 .843 (.028) .985 (.012) .939 (.024) .969 (.014) .962 (.016) .938 (.025) .970 (.014)
500 .845 (.017) .976 (.008) .933 (.015) .969 (.008) .967 (.008) .934 (.015) .970 (.008)
1000 .846 (.012) .973 (.005) .931 (.011) .970 (.005) .969 (.005) .932 (.011) .971 (.005)
1,000,000 .846 .970 .927 .970 .971 .928 .971
Asymmetric 100 .830 (.043) .990 (.022) .928 (.049) .956 (.033) .866 (.073) .915 (.068) .950 (.043)
200 .843 (.029) .989 (.015) .940 (.031) .966 (.020) .922 (.034) .935 (.037) .964 (.023)
500 .846 (.018) .981 (.011) .940 (.018) .969 (.011) .953 (.014) .939 (.020) .969 (.011)
1000 .846 (.012) .976 (.007) .938 (.013) .970 (.007) .962 (.008) .938 (.013) .970 (.007)
1,000,000 .846 .969 .932 .969 .970 .934 .970
Mixed 100 .833 (.044) .999 (.004) .994 (.018) .995 (.014) .887 (.052) .999 (.007) .999 (.005)
200 .842 (.029) .999 (.004) .992 (.014) .993 (.011) .923 (.027) .999 (.005) .999 (.004)
500 .845 (.017) .986 (.009) .966 (.013) .976 (.009) .941 (.012) .976 (.012) .989 (.010)
1000 .846 (.012) .976 (.007) .957 (.009) .971 (.007) .954 (.008) .961 (.010) .970 (.007)
1,000,000 .846 .970 .955 .970 .971 .957 .971
74
Table 14
.3 Symmetric 100 .496 (.058) .772 (.070) .596 (.088) .749 (.071) .742 (.085) .507 (.118) .712 (.086)
200 .495 (.045) .757 (.051) .574 (.062) .745 (.051) .729 (.062) .486 (.084) .714 (.062)
500 .491 (.029) .747 (.032) .555 (.038) .742 (.032) .722 (.039) .466 (.053) .716 (.039)
1000 .488 (.020) .743 (.023) .547 (.027) .740 (.023) .718 (.027) .457 (.037) .715 (.027)
1,000,000 .485 .738 .537 .738 .714 .445 .714
Asymmetric 100 .496 (.060) .773 (.072) .601 (.090) .749 (.073) .742 (.088) .515 (.120) .713 (.089)
200 .495 (.044) .756 (.052) .577 (.063) .744 (.052) .727 (.063) .491 (.085) .713 (.063)
500 .491 (.030) .746 (.034) .557 (.040) .741 (.034) .721 (.041) .470 (.055) .715 (.041)
1000 .488 (.021) .742 (.024) .548 (.028) .739 (.024) .717 (.029) .460 (.039) .714 (.029)
1,000,000 .485 .737 .539 .737 .714 .449 .714
Mixed 100 .497 (.061) .771 (.072) .600 (.091) .748 (.073) .741 (.087) .514 (.120) .712 (.088)
200 .494 (.045) .755 (.051) .576 (.062) .742 (.052) .726 (.062) .490 (.085) .712 (.063)
500 .491 (.029) .747 (.033) .560 (.039) .742 (.033) .722 (.039) .474 (.054) .716 (.039)
1000 .487 (.021) .741 (.023) .549 (.027) .738 (.023) .716 (.028) .461 (.038) .714 (.028)
1,000,000 .486 .737 .540 .737 .714 .451 .714
.7 Symmetric 100 .832 (.044) .987 (.013) .923 (.033) .968 (.016) .992 (.012) .924 (.035) .969 (.016)
200 .843 (.029) .979 (.010) .915 (.025) .969 (.011) .983 (.010) .915 (.026) .970 (.011)
500 .846 (.018) .974 (.006) .905 (.016) .969 (.006) .976 (.006) .906 (.017) .971 (.006)
1000 .846 (.012) .971 (.004) .899 (.012) .969 (.004) .973 (.004) .900 (.012) .971 (.004)
1,000,000 .847 .969 .894 .969 .971 .895 .971
Asymmetric 100 .832 (.043) .986 (.013) .924 (.032) .968 (.016) .991 (.013) .925 (.034) .969 (.016)
200 .841 (.029) .979 (.010) .915 (.024) .969 (.011) .982 (.010) .916 (.025) .969 (.011)
500 .845 (.018) .974 (.006) .906 (.016) .969 (.007) .975 (.006) .907 (.017) .970 (.006)
1000 .846 (.012) .972 (.004) .901 (.011) .969 (.004) .973 (.004) .902 (.012) .971 (.004)
1,000,000 .847 .969 .895 .969 .971 .896 .971
Mixed 100 .831 (.043) .986 (.014) .924 (.032) .967 (.016) .991 (.014) .925 (.034) .968 (.017)
200 .844 (.029) .980 (.010) .918 (.023) .969 (.010) .982 (.010) .918 (.024) .970 (.010)
500 .845 (.018) .974 (.006) .907 (.016) .969 (.007) .975 (.006) .908 (.017) .970 (.006)
1000 .846 (.013) .972 (.005) .902 (.012) .969 (.005) .973 (.004) .903 (.012) .971 (.005)
1,000,000 .846 .969 .897 .969 .971 .898 .971
75
Table 15
.3 Symmetric 100 .880 (.036) .990 (.016) .951 (.036) .975 (.020) .966 (.027) .939 (.046) .969 (.025)
200 .879 (.026) .984 (.013) .946 (.026) .974 (.014) .967 (.018) .936 (.032) .969 (.017)
500 .879 (.016) .977 (.008) .943 (.017) .973 (.009) .968 (.010) .934 (.019) .969 (.010)
1000 .879 (.011) .975 (.006) .942 (.012) .973 (.006) .969 (.007) .934 (.014) .970 (.007)
1,000,000 .878 .973 .940 .973 .969 .933 .969
Asymmetric 100 .880 (.036) .991 (.020) .954 (.045) .975 (.028) .923 (.054) .935 (.069) .964 (.040)
200 .880 (.026) .988 (.016) .952 (.032) .974 (.019) .944 (.033) .937 (.047) .966 (.028)
500 .879 (.016) .981 (.012) .949 (.021) .974 (.012) .960 (.017) .938 (.026) .968 (.015)
1000 .878 (.011) .977 (.008) .947 (.014) .973 (.008) .964 (.010) .938 (.017) .969 (.010)
1,000,000 .878 .973 .946 .973 .970 .939 .970
Mixed 100 .879 (.036) .998 (.009) .992 (.025) .994 (.018) .921 (.055) .993 (.028) .996 (.018)
200 .879 (.026) .998 (.008) .992 (.017) .993 (.013) .944 (.030) .992 (.021) .995 (.014)
500 .878 (.016) .989 (.010) .978 (.015) .983 (.011) .954 (.015) .977 (.021) .985 (.014)
1000 .879 (.011) .980 (.008) .968 (.011) .977 (.008) .959 (.011) .962 (.016) .973 (.010)
1,000,000 .878 .975 .964 .975 .970 .959 .970
76
Table 16
.3 Symmetric 100 .881 (.036) .985 (.013) .935 (.033) .975 (.015) .986 (.017) .920 (.043) .969 (.019)
200 .878 (.026) .980 (.009) .928 (.024) .974 (.010) .978 (.012) .913 (.029) .968 (.012)
500 .879 (.016) .976 (.006) .925 (.015) .974 (.006) .973 (.007) .910 (.019) .969 (.007)
1000 .878 (.011) .975 (.004) .923 (.011) .974 (.004) .971 (.005) .909 (.013) .969 (.005)
1,000,000 .878 .973 .921 .973 .970 .908 .970
Asymmetric 100 .881 (.037) .985 (.013) .936 (.033) .974 (.015) .985 (.017) .921 (.042) .968 (.019)
200 .879 (.025) .980 (.010) .930 (.023) .974 (.010) .978 (.012) .915 (.029) .969 (.012)
500 .879 (.016) .976 (.006) .925 (.015) .974 (.006) .973 (.007) .912 (.019) .969 (.007)
1000 .879 (.011) .975 (.004) .924 (.011) .974 (.004) .971 (.005) .910 (.013) .969 (.005)
1,000,000 .878 .974 .922 .974 .970 .909 .970
Mixed 100 .881 (.035) .985 (.013) .936 (.032) .974 (.015) .985 (.017) .921 (.041) .968 (.019)
200 .879 (.025) .980 (.010) .930 (.024) .974 (.010) .978 (.012) .916 (.029) .969 (.013)
500 .879 (.016) .976 (.006) .927 (.016) .974 (.006) .973 (.008) .913 (.019) .969 (.008)
1000 .878 (.012) .975 (.005) .925 (.011) .974 (.005) .971 (.005) .912 (.013) .969 (.005)
1,000,000 .878 .973 .923 .973 .969 .910 .969
77
Table 17
.3 Symmetric 100 .892 (.028) .991 (.014) .943 (.031) .967 (.020) .940 (.030) .935 (.039) .962 (.024)
200 .902 (.017) .983 (.012) .940 (.022) .968 (.013) .952 (.018) .931 (.027) .963 (.016)
500 .904 (.010) .974 (.007) .936 (.013) .968 (.008) .960 (.009) .928 (.016) .964 (.009)
1000 .904 (.007) .971 (.005) .935 (.010) .968 (.005) .962 (.006) .926 (.011) .964 (.006)
1,000,000 .905 .968 .932 .968 .965 .924 .965
Asymmetric 100 .893 (.027) .992 (.016) .942 (.038) .964 (.026) .843 (.063) .924 (.059) .952 (.039)
200 .902 (.017) .988 (.014) .944 (.027) .967 (.017) .901 (.037) .927 (.040) .956 (.026)
500 .904 (.010) .979 (.010) .943 (.016) .969 (.010) .941 (.016) .933 (.020) .962 (.013)
1000 .904 (.007) .974 (.007) .942 (.011) .969 (.007) .952 (.009) .933 (.013) .963 (.008)
1,000,000 .904 .969 .939 .969 .965 .932 .965
Mixed 100 .893 (.027) .999 (.006) .987 (.026) .990 (.020) .831 (.061) .996 (.017) .997 (.012)
200 .902 (.017) .998 (.007) .986 (.019) .988 (.016) .889 (.031) .989 (.022) .993 (.015)
500 .904 (.010) .985 (.010) .968 (.014) .975 (.011) .929 (.014) .969 (.018) .978 (.013)
1000 .905 (.007) .976 (.007) .960 (.009) .971 (.007) .945 (.009) .957 (.012) .969 (.009)
1,000,000 .905 .970 .958 .970 .965 .951 .965
78
Table 18
.3 Symmetric 100 .892 (.028) .985 (.013) .927 (.028) .968 (.014) .986 (.016) .915 (.036) .962 (.018)
200 .902 (.017) .978 (.009) .921 (.021) .969 (.010) .976 (.012) .908 (.025) .963 (.012)
500 .904 (.010) .973 (.006) .915 (.013) .969 (.006) .969 (.007) .902 (.016) .964 (.007)
1000 .905 (.007) .971 (.004) .912 (.009) .969 (.004) .967 (.005) .899 (.011) .964 (.005)
1,000,000 .904 .968 .908 .968 .964 .895 .964
Asymmetric 100 .891 (.028) .985 (.013) .927 (.028) .967 (.014) .985 (.016) .916 (.035) .962 (.018)
200 .902 (.017) .979 (.009) .923 (.020) .969 (.009) .976 (.011) .911 (.024) .964 (.011)
500 .904 (.010) .973 (.006) .915 (.013) .969 (.006) .969 (.007) .903 (.016) .964 (.007)
1000 .904 (.007) .971 (.004) .913 (.009) .969 (.004) .967 (.005) .900 (.011) .964 (.005)
1,000,000 .905 .969 .909 .969 .964 .896 .964
Mixed 100 .893 (.027) .985 (.013) .930 (.027) .968 (.014) .985 (.016) .918 (.035) .963 (.018)
200 .902 (.017) .978 (.009) .923 (.019) .968 (.009) .975 (.011) .911 (.023) .963 (.011)
500 .904 (.010) .972 (.006) .917 (.013) .969 (.006) .969 (.007) .904 (.015) .964 (.007)
1000 .904 (.007) .971 (.004) .914 (.009) .969 (.004) .967 (.005) .901 (.011) .964 (.005)
1,000,000 .905 .969 .911 .969 .965 .898 .965
79
Table 19
.3 Symmetric 100 .430 (.068) .759 (.091) .600 (.109) .715 (.091) .651 (.105) .518 (.140) .673 (.110)
200 .428 (.051) .730 (.070) .575 (.082) .707 (.070) .663 (.081) .506 (.105) .676 (.083)
500 .422 (.034) .711 (.044) .553 (.051) .702 (.044) .672 (.052) .490 (.066) .677 (.052)
1000 .420 (.024) .704 (.032) .543 (.036) .699 (.032) .674 (.037) .481 (.047) .677 (.037)
1,000,000 .419 .699 .534 .699 .679 .473 .679
Asymmetric 100 .432 (.068) .806 (.102) .642 (.121) .745 (.100) .546 (.128) .494 (.185) .654 (.140)
200 .429 (.050) .757 (.083) .604 (.097) .722 (.082) .589 (.107) .483 (.141) .650 (.111)
500 .421 (.033) .726 (.053) .586 (.060) .711 (.053) .639 (.067) .499 (.084) .666 (.068)
1000 .420 (.024) .716 (.038) .581 (.042) .708 (.038) .658 (.048) .505 (.058) .673 (.048)
1,000,000 .418 .706 .573 .706 .679 .506 .679
Mixed 100 .428 (.068) .879 (.106) .760 (.135) .798 (.116) .563 (.117) .810 (.165) .845 (.138)
200 .426 (.051) .772 (.089) .684 (.101) .729 (.089) .604 (.089) .696 (.114) .739 (.099)
500 .423 (.033) .715 (.055) .626 (.063) .699 (.055) .637 (.057) .643 (.068) .689 (.059)
1000 .421 (.023) .697 (.037) .605 (.042) .690 (.037) .654 (.040) .605 (.050) .677 (.040)
1,000,000 .419 .686 .595 .686 .677 .583 .677
.7 Symmetric 100 .808 (.050) .991 (.016) .931 (.037) .961 (.024) .946 (.032) .931 (.041) .963 (.025)
200 .822 (.032) .983 (.014) .931 (.027) .965 (.016) .957 (.018) .930 (.028) .965 (.016)
500 .825 (.020) .973 (.009) .925 (.017) .965 (.009) .963 (.009) .925 (.017) .966 (.009)
1000 .825 (.014) .970 (.006) .922 (.012) .966 (.006) .965 (.006) .922 (.012) .967 (.006)
1,000,000 .825 .966 .917 .966 .967 .919 .967
Asymmetric 100 .807 (.049) .988 (.025) .918 (.055) .950 (.038) .848 (.083) .903 (.077) .943 (.049)
200 .822 (.033) .988 (.017) .932 (.035) .962 (.022) .912 (.039) .926 (.042) .959 (.026)
500 .825 (.020) .978 (.012) .932 (.021) .965 (.012) .946 (.016) .931 (.022) .965 (.013)
1000 .826 (.014) .972 (.008) .929 (.014) .966 (.008) .957 (.009) .930 (.015) .966 (.008)
1,000,000 .825 .965 .923 .965 .967 .925 .967
Mixed 100 .811 (.049) .999 (.004) .993 (.020) .995 (.016) .872 (.059) .999 (.008) .999 (.006)
200 .821 (.033) .999 (.004) .991 (.016) .993 (.013) .913 (.030) .999 (.006) .999 (.005)
500 .825 (.019) .984 (.010) .962 (.015) .972 (.011) .933 (.014) .973 (.013) .977 (.011)
1000 .826 (.014) .973 (.008) .951 (.011) .967 (.008) .947 (.009) .956 (.011) .966 (.008)
1,000,000 .826 .966 .948 .966 .967 .952 .967
80
Table 20
.3 Symmetric 100 .429 (.066) .741 (.079) .542 (.100) .715 (.080) .707 (.097) .442 (.134) .673 (.097)
200 .428 (.051) .725 (.058) .517 (.070) .711 (.058) .693 (.070) .418 (.095) .676 (.071)
500 .423 (.032) .713 (.036) .496 (.043) .708 (.036) .685 (.044) .395 (.060) .678 (.044)
1000 .420 (.023) .709 (.026) .487 (.030) .706 (.026) .681 (.031) .385 (.042) .678 (.031)
1,000,000 .417 .703 .475 .703 .676 .371 .676
Asymmetric 100 .429 (.068) .742 (.082) .548 (.102) .716 (.082) .707 (.100) .450 (.137) .675 (.101)
200 .428 (.050) .724 (.059) .520 (.071) .710 (.059) .691 (.072) .423 (.096) .675 (.072)
500 .423 (.034) .712 (.038) .498 (.045) .707 (.038) .683 (.046) .400 (.062) .677 (.046)
1000 .419 (.024) .707 (.027) .488 (.032) .704 (.027) .680 (.032) .388 (.044) .676 (.032)
1,000,000 .417 .702 .478 .702 .676 .376 .676
Mixed 100 .430 (.070) .741 (.082) .547 (.103) .715 (.083) .706 (.099) .450 (.135) .674 (.100)
200 .426 (.051) .722 (.058) .519 (.071) .708 (.059) .689 (.071) .422 (.096) .673 (.071)
500 .424 (.033) .714 (.037) .501 (.044) .708 (.037) .685 (.044) .404 (.061) .679 (.045)
1000 .419 (.024) .707 (.027) .489 (.031) .704 (.027) .679 (.032) .389 (.043) .676 (.032)
1,000,000 .417 .702 .479 .702 .676 .378 .676
.7 Symmetric 100 .810 (.050) .985 (.015) .912 (.037) .963 (.018) .991 (.014) .913 (.039) .965 (.018)
200 .822 (.033) .977 (.011) .904 (.028) .965 (.012) .981 (.012) .904 (.029) .966 (.012)
500 .825 (.020) .970 (.007) .893 (.018) .965 (.007) .973 (.007) .893 (.019) .967 (.007)
1000 .825 (.014) .968 (.005) .886 (.013) .965 (.005) .970 (.005) .887 (.014) .967 (.005)
1,000,000 .826 .965 .879 .965 .967 .881 .967
Asymmetric 100 .809 (.049) .985 (.015) .914 (.036) .963 (.018) .990 (.015) .915 (.038) .964 (.019)
200 .820 (.032) .976 (.011) .904 (.027) .964 (.012) .979 (.012) .905 (.028) .965 (.012)
500 .825 (.020) .970 (.007) .893 (.018) .965 (.007) .972 (.007) .894 (.019) .966 (.007)
1000 .825 (.014) .968 (.005) .888 (.013) .965 (.005) .969 (.005) .899 (.013) .967 (.005)
1,000,000 .826 .965 .881 .965 .967 .882 .967
Mixed 100 .809 (.049) .984 (.016) .914 (.036) .963 (.019) .989 (.015) .915 (.039) .964 (.019)
200 .823 (.033) .977 (.011) .907 (.026) .965 (.012) .980 (.011) .907 (.028) .966 (.012)
500 .825 (.020) .970 (.007) .895 (.018) .965 (.007) .972 (.007) .896 (.019) .966 (.007)
1000 .825 (.014) .968 (.005) .889 (.013) .965 (.005) .969 (.005) .891 (.014) .967 (.005)
1,000,000 .826 .965 .883 .965 .967 .884 .967
81
Table 21
.3 Symmetric 100 .820 (.054) .985 (.023) .927 (.054) .963 (.030) .948 (.040) .909 (.069) .954 (.038)
200 .819 (.039) .975 (.020) .919 (.040) .960 (.021) .950 (.027) .903 (.048) .953 (.025)
500 .818 (.023) .966 (.013) .914 (.025) .960 (.013) .953 (.015) .901 (.029) .954 (.015)
1000 .818 (.017) .963 (.009) .913 (.018) .960 (.009) .954 (.010) .901 (.020) .954 (.010)
1,000,000 .816 .959 .910 .959 .954 .899 .954
Asymmetric 100 .819 (.054) .987 (.030) .931 (.068) .962 (.042) .885 (.082) .903 (.104) .947 (.061)
200 .820 (.038) .981 (.024) .928 (.049) .962 (.029) .916 (.050) .905 (.070) .949 (.042)
500 .818 (.024) .971 (.017) .924 (.031) .961 (.018) .939 (.025) .908 (.039) .952 (.023)
1000 .818 (.017) .965 (.012) .921 (.021) .960 (.012) .947 (.015) .907 (.026) .953 (.015)
1,000,000 .817 .960 .919 .960 .954 .908 .954
Mixed 100 .818 (.054) .998 (.013) .988 (.037) .991 (.027) .882 (.082) .990 (.042) .993 (.027)
200 .819 (.039) .996 (.012) .987 (.025) .990 (.020) .916 (.045) .988 (.032) .992 (.021)
500 .818 (.024) .984 (.015) .967 (.022) .975 (.017) .932 (.023) .965 (.032) .977 (.020)
1000 .818 (.017) .970 (.012) .952 (.017) .965 (.013) .939 (.016) .943 (.024) .960 (.016)
1,000,000 .817 .962 .946 .962 .955 .938 .955
82
Table 22
.3 Symmetric 100 .821 (.054) .978 (.020) .903 (.050) .962 (.022) .979 (.025) .880 (.064) .953 (.028)
200 .818 (.038) .969 (.014) .892 (.035) .961 (.015) .967 (.018) .970 (.044) .953 (.018)
500 .818 (.024) .964 (.009) .887 (.023) .961 (.009) .959 (.011) .866 (.028) .954 (.011)
1000 .817 (.017) .962 (.006) .885 (.016) .960 (.006) .957 (.007) .864 (.019) .954 (.007)
1,000,000 .817 .960 .882 .960 .954 .862 .954
Asymmetric 100 .821 (.055) .978 (.020) .904 (.050) .961 (.022) .978 (.025) .882 (.063) .953 (.028)
200 .819 (.038) .970 (.014) .895 (.035) .961 (.015) .967 (.018) .873 (.044) .953 (.018)
500 .818 (.024) .964 (.009) .888 (.023) .960 (.009) .959 (.011) .868 (.028) .954 (.011)
1000 .818 (.017) .962 (.006) .886 (.016) .960 (.007) .957 (.008) .866 (.020) .954 (.008)
1,000,000 .817 .960 .884 .960 .955 .864 .955
Mixed 100 .821 (.053) .977 (.020) .904 (.048) .961 (.022) .977 (.025) .882 (.061) .952 (.028)
200 .818 (.038) .970 (.015) .896 (.036) .961 (.016) .966 (.019) .874 (.044) .953 (.019)
500 .818 (.025) .964 (.010) .890 (.023) .961 (.010) .959 (.011) .870 (.029) .954 (.012)
1000 .817 (.018) .962 (.007) .887 (.017) .960 (.007) .957 (.008) .868 (.020) .954 (.008)
1,000,000 .817 .960 .884 .960 .954 .865 .954
83
Table 23
.3 Symmetric 100 .875 (.032) .990 (.016) .934 (.036) .962 (.023) .931 (.034) .924 (.046) .956 (.028)
200 .886 (.020) .981 (.014) .930 (.026) .963 (.015) .944 (.021) .920 (.031) .957 (.018)
500 .889 (.011) .970 (.009) .926 (.016) .963 (.009) .953 (.011) .916 (.018) .958 (.010)
1000 .889 (.008) .967 (.006) .923 (.011) .963 (.006) .956 (.007) .915 (.013) .959 (.007)
1,000,000 .889 .963 .921 .963 .959 .912 .959
Asymmetric 100 .876 (.032) .991 (.019) .932 (.044) .958 (.030) .817 (.073) .912 (.068) .945 (.045)
200 .886 (.020) .987 (.016) .935 (.031) .962 (.020) .885 (.043) .915 (.046) .949 (.031)
500 .889 (.011) .976 (.011) .934 (.019) .963 (.012) .931 (.018) .922 (.024) .956 (.015)
1000 .889 (.008) .970 (.008) .932 (.008) .964 (.008) .945 (.011) .922 (.016) .957 (.010)
1,000,000 .889 .964 .930 .964 .959 .921 .959
Mixed 100 .876 (.031) .999 (.007) .985 (.030) .988 (.023) .804 (.070) .995 (.020) .997 (.014)
200 .886 (.020) .998 (.008) .983 (.022) .986 (.018) .871 (.036) .987 (.025) .992 (.017)
500 .889 (.012) .983 (.012) .963 (.016) .971 (.013) .918 (.017) .964 (.021) .975 (.015)
1000 .890 (.008) .972 (.008) .954 (.011) .966 (.008) .937 (.010) .950 (.014) .964 (.010)
1,000,000 .890 .965 .951 .965 .959 .944 .959
84
Table 24
.3 Symmetric 100 .875 (.032) .983 (.015) .915 (.033) .963 (.017) .984 (.019) .902 (.042) .956 (.021)
200 .886 (.020) .975 (.011) .909 (.024) .964 (.011) .972 (.013) .894 (.029) .958 (.014)
500 .889 (.011) .968 (.007) .902 (.015) .964 (.007) .964 (.008) .886 (.018) .958 (.008)
1000 .890 (.008) .966 (.005) .898 (.011) .964 (.005) .962 (.006) .883 (.013) .959 (.006)
1,000,000 .889 .963 .893 .963 .959 .878 .959
Asymmetric 100 .874 (.032) .982 (.015) .916 (.032) .962 (.017) .982 (.019) .902 (.040) .956 (.021)
200 .886 (.020) .975 (.010) .911 (.023) .964 (.011) .972 (.013) .897 (.028) .958 (.013)
500 .889 (.011) .968 (.007) .902 (.015) .963 (.007) .964 (.008) .887 (.018) .958 (.008)
1000 .889 (.008) .966 (.005) .899 (.011) .964 (.005) .961 (.006) .884 (.013) .959 (.006)
1,000,000 .889 .964 .895 .964 .959 .880 .959
Mixed 100 .876 (.031) .983 (.015) .918 (.032) .963 (.017) .983 (.019) .905 (.040) .957 (.021)
200 .886 (.020) .975 (.010) .911 (.022) .963 (.011) .971 (.013) .897 (.027) .958 (.013)
500 .889 (.011) .968 (.007) .904 (.015) .964 (.007) .964 (.008) .889 (.018) .958 (.008)
1000 .889 (.008) .966 (.005) .900 (.011) .964 (.005) .961 (.006) .885 (.013) .958 (.006)
1,000,000 .890 .964 .897 .964 .959 .882 .959
85
APPENDIX C
INADMISSIBLE SOLUTION RATES AT THE SAMPLE LEVEL
Table 25
Inadmissible Solution Rates (%) for Model 1 With 9 Items
ρ Threshold n 2-point Scale 4-point Scale
ML WLSMV ULSMV ML WLSMV ULSMV
.3 Symmetric 100 0.0 .9 1.0 0.0 0.0 .1
200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Asymmetric 100 0.0 10.4 15.0 0.0 0.0 0.0

200 0.0 .6 1.3 0.0 0.0 0.0
500 0.0 0.0 0 0.0 0.0 0.0
1000 0.0 0.0 0 0.0 0.0 0.0
Mixed 100 0.0 12.2 10.5 0.0 0.0 0.0

200 0.0 1.2 .8 0.0 0.0 0.0
500 0.0 0.0 0 0.0 0.0 0.0
1000 0.0 0.0 0 0.0 0.0 0.0
.7 Symmetric 100 0.0 0.0 0 0.0 0.0 0.0

200 0.0 0.0 0 0.0 0.0 0.0
500 0.0 0.0 0 0.0 0.0 0.0
1000 0.0 0.0 0 0.0 0.0 0.0
Asymmetric 100 0.0 1.6 1.7 0.0 0.0 0.0

200 0.0 0.0 0 0.0 0.0 0.0
500 0.0 0.0 0 0.0 0.0 0.0
1000 0.0 0.0 0 0.0 0.0 0.0
Mixed 100 0.0 .9 .5 0.0 0.0 0.0

200 0.0 .1 0 0.0 0.0 0.0
500 0.0 0.0 0 0.0 0.0 0.0
1000 0.0 0.0 0 0.0 0.0 0.0
86
Table 26
.3 Symmetric 100 0.0 0.0 0.0 0.0 0.0 0.0
200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Asymmetric 100 0.0 .4 .5 0.0 0.0 0.0

200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Mixed 100 0.0 .3 0.0 0.0 0.0 0.0

200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
.7 Symmetric 100 0.0 0.0 0.0 0.0 0.0 0.0

200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Asymmetric 100 0.0 .3 .1 0.0 0.0 0.0

200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Mixed 100 0.0 0.0 0.0 0.0 0.0 0.0

200 0.0 0.0 0.0 0.0 0.0 0.0
500 0.0 0.0 0.0 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
87
Table 27
.3 Symmetric 100 2.8 25.8 32.5 3.1 3.2 6.1
200 0.0 4.8 8.2 0.0 .1 .3
500 0.0 .1 .4 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Asymmetric 100 2.3 54.3 63.6 2.1 3.2 6.4

200 0.0 20.6 30.1 .1 .2 .5
500 0.0 1.2 3.7 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Mixed 100 2.7 58.8 52.4 2.6 4.1 7.8

200 .1 24.8 22.0 0.0 .1 .6
500 0.0 3.9 6.7 0.0 0.0 0.0
1000 0.0 .3 1.3 0.0 0.0 0.0
88
Table 28
.3 Symmetric 100 0.0 23.1 35.9 0.0 2.4 8.9
200 0.0 4.9 11.8 0.0 .2 .9
500 0.0 .6 .7 0.0 0.0 0.0
1000 0.0 0.0 0.0 0.0 0.0 0.0
Asymmetric 100 0.0 43.1 57.6 0.0 4.3 11.5

200 0.0 16.4 33.4 0.0 0.0 1.0
500 0.0 .8 5.7 0.0 0.0 0.0
1000 0.0 0.0 .3 0.0 0.0 0.0
Mixed 100 0.0 34.7 15.5 0.0 4.5 11.0

200 0.0 36.7 12.4 0.0 .3 1.2
500 0.0 25.1 7.3 0.0 0.0 0.0
1000 0.0 8.5 2.1 0.0 0.0 0.0
89
REFERENCES
Asparouhov, T., & Muthén, B. (2010). Simple second order chi-square correction. Retrieved
from http://www.statmodel.com/download/WLSMV new chi21.pdf
Ayer, L., Althoff, R., Ivanova, M., Rettew, D., Waxler, E., Sulman, J., & Hudziak, J. (2009).
Child Behavior Checklist Juvenile Bipolar Disorder (CBCL‐JBD) and CBCL
Posttraumatic Stress Problems (CBCL‐PTSP) scales are measures of a single
dysregulatory syndrome. Journal of Child Psychology and Psychiatry, 50, 1291-1300.
Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus
means and variance adjusted weighted least squares estimation in CFA. Structural
Equation Modeling: A Multidisciplinary Journal, 13, 186-203.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107,
238-246.
Bentler, P. M. (2008). EQS structural equation modeling software. Encino, CA: Multivariate
Software.
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of
covariance structures. Psychological Bulletin, 88, 588–606.
Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
Brosseau-Liard, P. E., & Savalei, V. (2014). Adjusting incremental fit indexes for
nonnormality. Multivariate Behavioral Research, 49, 460-470.
Brosseau-Liard, P. E., Savalei, V., & Li, L. (2012). An investigation of the sample performance
of two nonnormality corrections for RMSEA. Multivariate Behavioral Research, 47,
904-930.
Cudeck, R., & Browne, M. W. (1992). Constructing a covariance matrix that yields a specified
minimizer and a specified minimum discrepancy function value. Psychometrika, 57, 357-
369.
Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality
and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29.
Currier, J. M., & Holland, J. M. (2014). Involvement in abusive violence among Vietnam
veterans: Direct and indirect associations with substance use problems and suicidality.
Psychological Trauma: Theory, Research, Practice, and Policy, 6, 73-82.
90
De Beer, L. T., Pienaar, J., & Rothmann, S. (2014). Job burnout’s relationship with sleep
difficulties in the presence of control variables: a self-report study. South African Journal
of Psychology, 44, 454-466.
DiStefano, C., & Morgan, G. B. (2014). A comparison of diagonal weighted least squares robust
estimation techniques for ordinal data. Structural Equation Modeling: A
Multidisciplinary Journal, 21, 425-438.
Eaton, N. R., Krueger, R. F., Markon, K. E., Keyes, K. M., Skodol, A. E., Wall, M., ... & Grant,
B. F. (2013). The structure and predictive validity of the internalizing disorders. Journal
of Abnormal Psychology, 122, 86-92.
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of

estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9,
466-491.
Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal
indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural
Garrido, L. E., Abad, F. J., & Ponsoda, V. (2015). Are Fit Indexes Really Fit to Estimate the
Number of Factors With Categorical Variables? Some Cautionary Findings via Monte
Carlo Simulation. Psychological Methods. http://dx.doi.org/10.1037/met0000064
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling: A
Multidisciplinary Journal, 6, 1-55.
Jöreskog, K. G. (1994). On the estimation of polychoric correlations and their asymptotic

covariance matrix. Psychometrika, 59, 381-389.
Jöreskog, K., & Sörbom, D. (1996). PRELIS 2: User’s reference guide. Chicago, IL: Scientific
Software International.
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and
Psychological Measurement, 20, 141-151.
Keyes, K. M., Eaton, N. R., Krueger, R. F., Skodol, A. E., Wall, M. M., Grant, B., ... & Hasin, D.
S. (2013). Thought disorder in the meta-structure of psychopathology. Psychological
Medicine, 43, 1673-1683.
Koziol, N. A. (2010). Evaluating measurement invariance with censored ordinal data: A Monte
Carlo comparison of alternative model estimators and scales of measurement. University
of Nebraska. Unpublished Master’s Thesis.
91
Li, C. H. (2014). The performance of MLR, USLMV, and WLSMV estimation in structural
regression models with ordinal variables (Doctoral dissertation, MICHIGAN STATE
UNIVERSITY).
Li, L., & Bentler, P. (2006). Robust statistical tests for evaluating the hypothesis of close fit of
misspecified mean and covariance structural models. UCLA Statistics Reprint #506. Los
Angeles: University of California.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-
testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing
Hu and Bentler's (1999) findings. Structural Equation Modeling: A Multidisciplinary
Journal, 11, 320-341.
McDonald, R. P., & Ho, M. H. R. (2002). Principles and practice in reporting structural equation
analyses. Psychological Methods, 7, 64-82.
Muthén, B. (1978). Contributions to factor analysis of dichotomous variables. Psychometrika,

43, 551-560.
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical,
and continuous latent variable indicators. Psychometrika, 49, 115–132.
Muthén, B. (1993). Goodness of fit with categorical and other non-normal variables. In K. A.
Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 205-243). Newbury
Park, CA: Sage.
Muthén, B. O., Du Toit, S., & Spisic, D. (1997). Robust inference using weighted least squares
and quadratic estimating equations in latent variable modeling with categorical and
continuous outcomes. Unpublished manuscript.
Muthén, L. K., & Muthén, B. O. (1998-2015). Mplus user’s guide (7th ed.). Los Angeles, CA:
Muthén & Muthén.
Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient.

Psychometrika, 44, 443-460.
Oranje, A. (2003). Comparison of estimation methods in factor analysis with categorized

variables: Applications to NAEP data. Paper presented at the annual meeting of the
American Educational Research Association, Chicago, IL.
Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When can categorical variables be
treated as continuous? A comparison of robust continuous and categorical SEM
estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373.
SAS Institute Inc. (2015). SAS/STAT 14.1 User’s Guide. Cary, NC: Author.
92
Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014). Evaluating model fit with ordered
categorical data with a measurement invariance framework: A comparison of estimators.
Structural Equation Modeling: A Multidisciplinary Journal, 21, 167-180.
Satorra, A. (1992). Asymptotic robust inferences in the analysis of mean and covariance
structures. In P. V. Marsden (Ed.), Sociological methodology. Oxford & Cambridge, MA:
Basil Blackwell.
Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in
covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables
analysis: Applications for developmental research. (pp. 399-419). Thousand Oaks, CA:
Sage.
Savalei, V. (2014). Understanding robust corrections in structural equation modeling. Structural

Savalei, V., & Rhemtulla, M. (2013). The performance of robust test statistics with categorical
data. British Journal of Mathematical and Statistical Psychology, 66, 201-223.
Stander, F. W., Mostert, K., & de Beer, L. T. (2014). Organisational and individual strengths use
as predictors of engagement and productivity. Journal of Psychology in Africa, 24, 403-
409.
Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation

approach. Multivariate Behavioral Research, 25, 173-180.
Steiger, J. H., & Lind, J. C. (1980). Statistically based tests for the number of common factors.
Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA.
Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor
analysis. Psychometrika, 38, 1–10.
Xia, Y., Yung, Y. F., & Zhang, W. (in press). Evaluating the selection of normal-theory weight
matrices in the Satorra-Bentler correction of chi-square and standard errors. Structural
Equation Modeling: A Multidisciplinary Journal.
Yang-Wallentin, F., Jöreskog, K. G., & Luo, H. (2010). Confirmatory factor analysis of ordinal
variables with misspecified models. Structural Equation Modeling: A Multidisciplinary
Journal, 17, 392-423.
Yang, Y., & Xia, Y. (2015). On the number of factors in exploratory factor analysis for ordered
categorical data. Behavior Research Methods, 47, 756-772.
Yu, C. Y. (2002). Evaluating cutoff criteria of model-fit indexes for latent variable models with
binary and continuous outcomes (Doctoral dissertation, University of California Los
Angeles).
93
Yuan, K-H, & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance
structure analysis with nonnormal missing data. Sociological Methodology, 30, 165-200.
94
BIOGRAPHICAL SKETCH
Yan Xia started his undergraduate study in Psychology in the Department of Psychology at
Peking University in China in 2006. Following completion of his Bachelor of Science degree in
2010, he enrolled in the Measurement and Statistics Ph.D. program in the Department of
Educational Psychology and Learning Systems at the Florida State University. He also dual-
enrolled in the MS program in the Department of Statistics at Florida State University in 2012
and completed the MS degree in 2015.
During his doctoral study, he has been a teaching assistant for a graduate course, a graduate
research assistant at the Center of Advancement of Learning and Assessment, a statistical
consultant for the College of Education, a tutorial assistant for the Center for Academic
Retention and Enhancement, and a statistics intern at the SAS Institute, Inc. His research interest
includes quantitative studies in structural equation modeling and item response theory.
95

FSU 2016SU Xia Fsu 0071E 13379

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

FSU 2016SU Xia Fsu 0071E 13379

Uploaded by

Copyright:

Available Formats

Florida State University Libraries

Investigating the Chi-Square-Based Model-

INVESTIGATING THE CHI-SQUARE-BASED MODEL-FIT INDEXES FOR

WLSMV AND ULSMV ESTIMATORS

A Dissertation submitted to the

Words are powerless to express my deepest gratitude to my communities of scholars,

List of Tables ................................................................................................................................ vii

WLSMV and ULSMV Estimators ..............................................................................................2

Confirmatory Factor Analysis with Ordered Categorical Variables .........................................10

Simulation at the Population Level ...........................................................................................26

Results at the Population Level .................................................................................................31

DISCUSSION AND CONCLUSIONS .........................................................................................49

The Dependency of Model Fit Indexes on Thresholds .............................................................50

A. RESULTS FROM THE SIMULATION AT THE POPULATION LEVEL............................55

25 Inadmissible Solution Rates (%) for Model 1 with 9 Items.....................................................86

26 Inadmissible Solution Rates (%) for Model 1 with 18 Items...................................................87

27 Inadmissible Solution Rates (%) for Model 2 with 9 Items.....................................................88

28 Inadmissible Solution Rates (%) for Model 2 with 18 Items...................................................89

WLSMV and ULSMV Estimators

Population-corrected and Sample-corrected Robust Model Fit Indexes

Confirmatory Factor Analysis with Ordered Categorical Variables

Confirmatory Factor Analysis Model

Threshold and Polychoric Correlation

(note that a0  b0   and as  br   ).

cumulative distribution function. The thresholds are estimated as

observed scores of x = i and y = j. The sample likelihood can then be expressed as

and 2 be the corresponding probability density function. It follows that

Estimation Methods for CFA with Ordered Categorical Data

FWLS   s  σ  θ   W1  s  σ  θ   . (4)

where Δ  σ  θ  / θ . Because in equation (5), W doesn’t have to be Γ , Muthén (1993)

Robust Corrections for the DWLS and ULS Chi-square Statistics

TWLSMV  aDWLS   n  1 FDWLS  bDWLS , (6)

TULSMV  aULS   n  1 FULS  bULS , (7)

where FULS is the fit function with an identity weight matrix,

Robust Model-fit Indexes for WLSMV and ULSMV

Chi-square-based Model-fit Indexes

model, and TB  (n  1) FˆB with TB  aB (n  1) FˆB  bB for the baseline model.

compared with aH  n  1 F H , and therefore RMSEAPR converges to

aH (n  1) FH / ((n  1)dH )  aH FH / dH , which is different from FH / d H . Similarly, CFIPR

converges to 1  aH (n 1) FH / (aB (n 1) FB )  1  aH FH / (aB FB ) and TLIPR converges to

1  aH (n  1) FH dB / (aB (n 1) FBdH )  1  aH FH dB / (aB FBdH ) .

Sample-corrected Robust Model-fit Indexes

In an unpublished article by Li and Bentler (2006), an alternative method was presented

Simulation at the Population Level

Two CFA models were employed for data generation.

Simulation at the Sample Level

Results at the Population Level

Results at the Sample Level

CFI and TLI

DISCUSSION AND CONCLUSIONS

The Dependency of Model Fit Indexes on Thresholds

The Accuracy of Model Fit Indexes

The Use of Conventional Cutoff Values

RESULTS FROM THE SIMULATION AT THE POPULATION LEVEL

RESULTS FROM THE SIMULATION AT THE SAMPLE LEVEL

INADMISSIBLE SOLUTION RATES AT THE SAMPLE LEVEL

Asymmetric 100 0.0 10.4 15.0 0.0 0.0 0.0

Mixed 100 0.0 12.2 10.5 0.0 0.0 0.0

.7 Symmetric 100 0.0 0.0 0 0.0 0.0 0.0

Asymmetric 100 0.0 1.6 1.7 0.0 0.0 0.0

Mixed 100 0.0 .9 .5 0.0 0.0 0.0

Asymmetric 100 0.0 .4 .5 0.0 0.0 0.0