Assessing unidimensionality of psychological scales : using individual and integrative criteria from factor analysis

Whenever one uses a composite scale score from item responses, one is tacitly assuming that the scale is dominantly unidimensional. Investigating the unidimensionality of item response data is an essential component of construct validity. Yet, there is no universally accepted technique or set of rules to determine the number of factors to retain when assessing the dimensionality of item response data. Typically factor analysis is used with the eigenvalues-greater- than-one rule, the ratio of first-to-second eigenvalues, parallel analysis (PA), root-mean-square- error-of-approximation (RMSEA), or hypothesis testing approaches involving chi-square tests from Maximum Likelihood (ML) or Generalized Least Squares (GLS) estimation. The purpose of this study was to investigate how these various procedures perform individually and in combination when assessing the unidimensionality of item response data via a computer simulated design. Conditions such as sample size, magnitude of communality, distribution of item responses, proportion of communality on second factor, and the number of items with nonzero loadings on the second factor were varied. Results indicate that there was no one individual decision-making method that identified undimensionality under all conditions manipulated. All individual decision-making methods failed to detect unidimensionality for the case where sample size was small, magnitude of communality was low, and item distributions were skewed. In addition, combination methods performed better than any one individual decision-making rule in certain sets of conditions. A set of guidelines and a new statistical methodology are provided for researchers. A future program of research is also illustrated.Education, Faculty ofEducational and Counselling Psychology, and Special Education (ECPS), Department ofGraduat