An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis

Session 3W - Posters: Written Assessment and Portfolio Assessment: abstract no.3W5BACKGROUND: Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language University in Hong Kong. SUMMARY OF WORK: Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options 1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. SUMMARY OF RESULTS: The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD=0.88) functioning distractors. Only 52.2% (n=805) of all distractors were functioning effectively and 10.2% (n=158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. CONCLUSIONS: The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. TAKE-HOME MESSAGES: Test items should consist of as many options as is feasible given the item content and the number of plausible distractors, in most cases this would be three.link_to_OA_fulltextConference of the Association for Medical Education in Europe (AMEE 2010), Glasgow, UK., 4-8 September 2010. In Abstract Book of AMEE, 2010, p. 12