Misfit: A Discussion and Some Alternatives for Existing Procedures

This article extends the work by Armstrong and Shi on CUmulative SUM (CUSUM) person-fit methodology. The authors present new theoretical considerations concerning the use of CUSUM person-fit statistics based on likelihood ratios for the purpose of detecting cheating and random guessing by individual test takers. According to the Neyman–Pearson Lemma, the optimality of such statistics relies on how accurately normal and aberrant behaviors are mod-eled. General and specific models for cheating and random guessing are investigated. The detec-tion rates of several statistics are compared using simulated data. Results showed that the likelihood-based CUSUM statistics that use the proposed models for aberrant behavior per-formed better than some of the more commonly used statistics, especially for cheating behavior. Keywords item response theory model, cheating, random guessing, aberrant behavior detection, likelihood ratio, cumulative sum The evaluation of individual test score validity is important in education and achievement test-ing. For example, a frequently encountered problem is the inflation of test scores due to pre