A Comparative Review of Skill Assessment: Performance, Prediction and Profiling

Abstract. The assessment of chess players is both an increasingly attractive op-portunity and an unfortunate necessity. The chess community needs to limit po-tential reputational damage by inhibiting cheating and unjustified accusations of cheating: there has been a recent rise in both. A number of counter-intuitive discoveries have been made by benchmarking the intrinsic merit of players’ moves: these call for further investigation. Is Capablanca actually, objectively the most accurate World Champion? Has ELO rating inflation not taken place? Stimulated by FIDE/ACP, we revisit the fundamentals of the subject to advance a framework suitable for improved standards of computational experiment and more precise results. Other games and domains look to chess as demonstrator of good practice, including the rating of professionals making high-value decisions under pressure, personnel evaluation by Multichoice Assessment and the organ-ization of crowd-sourcing in citizen science projects. The ‘3P ’ themes of per-formance, prediction and profiling pervade all these domains