Collaborative assessments in on-line classrooms

With massive open on-line courses (MOOCs) gaining momentum, it is now common for thousands of students to enrol in a course, making manual assessments by teachers simply unfeasible. Peer assessments is one way to go when auto-scoring approaches are not possible. Current on-line courses usually use a simple aggregation of peer assessments, but these suffer from two main pitfalls. First, simple aggregation does not take into consideration the reliability of a peer assessment. Second, simple aggregation calculates what the students think of an assignment as opposed to what the teacher thinks of it (the far more important opinion). This work proposes two different models to address these two different pitfalls. These models lay the foundation for future work, where we intend to combine both models into a single one that addresses both pitfalls at once. The aim is to build an automated assessment system that results from the collaboration of both students and teachers.The second author is supported by an Industrial PhD scholarship from the Generalitat de Catalunya. Furthermore, this work is supported by the Gencat 2014 SGR 118 project (funded by the Generalitat de Catalunya), and the Collective-Mind and Collectiveware projects (funded by the Spanish Ministry of Economy and Competitiveness, under grant numbers TEC2013-49430-EXP and TIN2015-66863-C2-1-R, respectively).Peer Reviewe