Investigating Bifactor Models and Fit Indices for Unidimensionality: An Illustration With Method Effects Due to Item Wording

The use of unidimensional scales that contain both positively and negatively worded items is common in both the educational and psychological fields. However, dimensionality investigations of these instruments often lead to a rejection of the theorized unidimensional model in favor of multidimensional structures, leaving researchers at odds for how best to treat their data. One modeling technique that has gained attention in recent years related to item wording effects is the bifactor model, which specifies a general construct that explains the covariation among all items, and two method factors that explain additional covariation among items with similar wording. Recent applications of the bifactor model have called for utilizing select model-based indices to establish “essentially unidimensional” models (i.e., Bonifay et al., 2015; Reise, Scheines, et al., 2013; Rodriguez et al., 2016a, 2016b). The purpose of this study was to investigate the performance of the select bifactor model-based indices, explained common variance (ECV), omega hierarchical (omegaH), and the percent of uncontaminated correlations (PUC), in establishing essentially unidimensional models. A Monte Carlo simulation study was conducted to examine the performance of these indices under conditions frequently encountered when item wording method effects are identified within unidimensional scales. Conditions included: 3 conditions related to the number of items per method factor (i.e., PUC values; 6:6, 8:4, 3:6) x 6 item loading values and patterns between the method factors and the general factor (0.7, 0.5 for the general factor; 0.5 or 0.3 for the method factors; four balanced and two unbalanced method factor conditions) x 2 item-level distributions (all normal; negative items non-normally distributed) x 2 model misspecification (correctly specified bifactor model, misspecified unidimensional model). Outcomes examined in this study included the relative bias in factor loadings estimated from a misspecified unidimensional model (i.e., ignoring the method factors) when data were simulated from a bifactor model. Results indicated the presence of unbalanced method factors (both in size and magnitude) required a substantially “stronger” general factor, as evidenced by higher ECV (\u3e0.80) and omegaH (\u3e0.85) values, to reach “essentially unidimensional” status. Furthermore, PUC was found to be relatively non-informative as an indicator of essential unidimensionality in the context of item wording method effects. Examination of the relationship between ECV and select model-fit indices indicated that while they are somewhat related (i.e., Reise, Scheines, et al., 2013), ECV, as well as omegaH, are more reliable as indices in identifying when data can be treated as essentially unidimensional in the context of item wording method effects. Recommendations and guidelines are provided for applied researchers utilizing unidimensional scales that may be contaminated by item wording method effects