Investigating Performance of Model Fit Indices In Multiple-Group Confirmatory Factor Analysis: Complications With Ordinal Data

The purpose of this study is to evaluate the performance of three commonly used model fit indices when measurement invariance is tested in the context of multiple-group CFA with categorical-ordered data. As applied researchers are increasingly aware of the importance of testing measurement invariance, as well as Likert-type scales are frequently used in the social and behavioral sciences, specific guidelines are in need for establishing measurement invariance using model fit indices. To achieve the study goal, two Monte Carlo simulation studies were conducted. Study 1 investigated the sampling variability of fit indices under different levels of invariance tests. Based on the sampling variability of fit indices, cutoff values for various levels of invariance were proposed. Study 2 investigated the influence of several conditions on the sensitivity of changes in fit indices to two commonly used non-invariance levels: metric non-invariance and scalar non-invariance. Then, rejection rates based on cutoff values of proposed fit indices were examined in Study 2. Findings indicated that all three fit indices (CFI, RMSEA, and SRMR) appeared to be more sensitive to lack of invariance in thresholds than loadings. Different cutoff values may be applied under various conditions with categorial-ordered data. In addition, cutoff values should be used with caution as factors impacted changes in model fit indices differently. Recommendations for the use of model fit indices in the multiple-group CFA invariance context were provided for applied researche