THE FIT OF EMPIRICAL DATA TO TWO LATENT TRAIT MODELS

The study explored fit of empirical data to the Rasch and three-parameter logistic latent trait models focusing upon the relationship between deviations from latent trait model assumptions and fit. The study also investigated estimation precision for small sample and short test conditions and evaluated parameter estimation costs for the two latent trait models. Rasch and three-parameter abilities and item parameters were estimated for twenty-five 40-item tests having 1000 examinees. These estimated parameters were substituted for true parameters to make predictions about number-correct score distributions using a theorem by Lord (1980) equating ability with the conditional distribution of number correct scores. Predicted score distributions were compared with observed score distributions by Kolmogorov-Smirnov and Chi square measures, and with graphical techniques. The importance of three latent trait model assumptions: unidimensionality, equality of item discrimination indices, and no guessing were assessed with correlation analyses. Estimation precision for short 20-item tests and for small samples of 250 examinees were evaluated with correlation methods and by assessing average absolute differences between estimates. Simple summary statistics were gathered to evaluate computer cost and time for parameter estimation with each model. The Rasch and the three-parameter models both demonstrated close fit to the majority of data studied. Eighty percent of tests fit both models quite well. Only one predicted test distribution deviated significantly from the observed score distribution. Results obtained with the Chi square measure were less favorable toward the models than Kolmogorov-Smirnov assessments had been. This outcome was attributed to the apparent sensitivity of the Chi square statistic to lack of normality in score distributions. Graphic results clearly supported statistical measures of fit leading to the conclusion that latent trait models adequately describe empirical test sets. The Rasch model fit data, overall, as well as the three-parameter model. The average K-S statistic for 25 tests was 1.304 for the Rasch model and 1.289 for the three-parameter model. The latter model fit data better than the Rasch model for 65 percent of the tests, yet the differences in fit between the models were insignificant. The Chi square measure and graphical tests supported these results. Lack of unidimensionality was the primary cause for misfit of data to the models. Correlations between fit statistics and indices of unidimensionality were significant at the .05 probability level for the Rasch and three-parameter models. When item discrimination parameters were unequal, fit of data to both models was impaired, and when guessing was present, while not well estimated on samples of 1000, fit of data to both latent trait models tended to be distorted. Ability estimates from short 20-item tests were quite precise, especially for the Rasch model. Correlations between ability estimates from the 20-item and longer tests were .923 for the Rasch estimates, and .866 for the three-parameter estimates. Difficulty estimates made from small 250 examinee samples were also quite precise, but estimates of other item parameters from small samples tended not to be very accurate. Although small sample item discrimination estimates were reasonable, estimates of the guessing parameter were very poor. The results suggest that at least 1000 examinees are required to obtain precise estimates with the three-parameter model. The average cost for estimating Rasch item parameters and abilities was only $12.50 for 1000 examinees in contrast to $35.12 for the three-parameter model, but when item parameters were known in advance, and only abilities estimated, these cost differences disappeared