The effects of ordinal data on coefficient alpha

Given coefficient alpha’s wide prevalence as a measure of internal reliability, it is important to know the conditions under which it is an appropriate estimate of reliability. The present paper explores alpha’s assumption of uncorrelated errors when used with ordinal data. Alpha overestimates true reliability when correlated errors are present. In this paper, I use a simulation study to recreate three mechanisms proposed to create correlated errors in ordinal data. The first mechanism, misclassification error, occurs when there are correlated measurement errors present in the data. The second mechanism, grouping error, occurs when there are not enough categories to represent the construct in question. The final mechanism is transformation error, which occurs when observed data do not match the distribution of true scores. Results indicated that misclassification and transformation error caused correlated errors, but only misclassification error caused correlated errors that were large enough for alpha to overestimate true reliability. Researchers should consider the assumption of correlated errors when reporting and making decisions based on alpha’s value alone