The effect of test length and IRT model on the distribution and stability of three appropriateness indices

The extent to which three appropriateness indexes-z&dquo; ECIZ4, and w (a variation of Wright’s person-fit statistic)-are well-standardized was investigated in a monte carlo study. To assess the effects of the item response theory (IRT.) model and test length on the distribution of the indexes and their cutoff values at three false positive rates, nonaberrant response patterns were generated. ECIZ4 most closely approximated a normal distribution, showing less skewness and kurtosis than z, and w. The ECIZ4 cutoff values were affected less by test length and the IRT model than were z, and w. In contrast, the distribution of w was the least stable over replications, and its cutoff values varied greatly depending on the IRT model and test length. Index terms: appropriateness measurement, caution index, item response theory (person fit), person-fit statistics, unusual response patterns. Valid interpretation of test scores is an im-portant concern in psychological and educational measurement, especially when test scores are used to make important decisions about individuals, because a test score may be invalid for an indi-vidual, even though the test may have satisfac-tory properties for the group being measured. Recently, quantitative measures called appro-priateness indexes have been developed to detect unusual examinee response patterns. A numbe