Assessing engineering students with multiple-choice exams: theoretical and empirical analysis of scoring methods

Multiple-choice questions (MPQ) are considered an objective way for testing large groups, and allow for fast feedback. A drawback of MPQ is that students can gain marks by guessing, and that depending on the marking method personality traits such as risk aversion might influence the total score. Therefore, a variety of marking methods for MPQ are available each with its own advantages and disadvantages. Methods such as negative marking try to discourage students from gaining marks by guessing by introducing a penalty for a wrong answer. Other methods such as standard setting correct for guessing by increasing the threshold for passing. Methods such as elimination marking, allow rewarding partial knowledge. This study tackles one important concern: the fair and objective marking of multiple-choice exams with a special focus to the influence of “guessing”. The focus is on comparing three widely used marking methods for MPQ: negative marking, standard setting, and elimination marking with adapted score rule (EMA). This paper is the first one to present results both theoretical and experimental related to EMA. Additionally, the paper has two contributions. Firstly, it introduces and uses a theoretical framework that combines statistics, economic, and psychometric models to study the effect of risk-aversion and ability for the different scoring methods. Secondly, it shows using an empirical analysis that the adapted elimination marking is a valid alternative for negative marking, and is preferred by students.status: publishe