The impact of student ability and method for varying the position of correct answers in classroom multiple-choice tests

Multiple-choice item-writing guideline research is in its infancy. Haladyna (2004) calls for a science of item-writing guideline research. The purpose of this study is to respond to such a call. The purpose of this study was to examine the impact of student ability and method for varying the location of correct answers in classroom multiple-choice tests. Educational testing literature supports the argument that randomizing the test-key is superior to other methods because it reduces the chances that test-takers can use guessing strategies successfully. However, the scant empirical literature on the impact of test-key formats has been restricted to large-scale educational tests. For this study, three test formats were developed for one test instrument based on different answer-placement strategies discussed in the educational measurement literature: a randomized, arbitrary, and balanced format. These test formats were randomly distributed to university students that participated in this study. Students were given an option to self-report which GPA range category they fell within. Based on these reports, students were placed into high, average, and low ability groups for analysis. Factorial ANOVA was conducted on the interaction and main effects of student ability and test format on test scores. Item analyses were also examined through indices of item difficulty, discrimination, and test reliability. Results showed no interaction effects between student ability and test format. Test scores differed across student ability as expected. Test scores were not significantly different across test formats. The argument to randomize the test-key over other methods in classroom multiple-choice tests requires more empirical attention given the evidence presented in this study