How to remove the gender bias in multiple choice assessments in engineering? Experimental validation and theoretical analysis using prospect theory

In higher education, exams with multiple choice questions (MPQ) are very common since they allow for testing large groups and giving fast feedback. While multiple choice exams are at first sight considered to be an objective way to assess knowledge of the students, the use of multiple choice exams raises some concerns. This research tackles one important concern: the fair and objective marking of multiple choice exams. A variety of marking methods are available such as number right, elimination testing, and negative marking. A method such as negative marking, which is the standard method at KU Leuven, tries to discourage blind guessing. Literature, both from the pedagogical and psychological field of “assessment” as well as the economic research field of “decision making under uncertainty”, has however indicated that negative marking disadvantages risk-averse students. Moreover, female students are in general more risk-averse. As such, introducing penalties for guessing can disadvantage female students. The aim of our research is to investigate if a method called “elimination testing” [1], originally designed to measure partial knowledge, is suited for grading multiple choice exams without gender and risk-averse bias in the field of engineering. To this end we did both an experimental and a theoretical investigation. For the experimental investigation the elimination testing was introduced in a first year Engineering Science bachelor course “Electrical circuits”. For the theoretical investigation we used prospect theory (a model from economics for decision making under uncertainty) [2,3] to compare the impact of different multiple choice methods on risk-averse students. Based on both investigations we conclude that elimination testing is not disadvantageous for risk-averse students. However, elimination testing results in a higher average score than the well-known negative marking. Therefore, test builders have to spend even more effort in the design of the question and the alternatives for the correct answer, since students are rewarded for partial knowledge. To alleviate this disadvantage we propose introduce a new scoring rule for elimination testing that is still gender neutral.status: publishe