Grading the Graders: Motivating Peer Graders in a MOOC

In this paper, we detail our efforts at creating and running a con-trolled study designed to examine how students in a MOOC might be motivated to do a better job during peer grading. This study in-volves more than one thousand students of a popular MOOC. We ask two specific questions: (1) When a student knows that his or her own peer grading efforts are being examined by peers, does this knowledge alone tend to motivate the student to do a better job when grading assignments? And (2) when a student not only knows that his or her own peer grading efforts are being examined by peers, but he or she is also given a number of other peer grading efforts to evaluate (so the peer graders see how other peer graders evaluate assignments), do both of these together tend to motivate the student to do a better job when grading assignments? We find strong statistical evidence that “grading the graders ” does in fact tend to increase the quality of peer grading. 1