1Computer Analysis of Essays

Research into the use of advanced computational linguistics techniques recently culminated in the implementation of a prototype automatic essay scoring system at Educational Testing Service (ETS). In an evaluation study using data sets from thirteen different GMAT essay prompts, this system, e-rater, showed between 87 % and 94 % agreement with expert readers ' scores, an accuracy comparable to that between two expert readers. This indicates that e-rater might be useful as a second reader for high stakes assessments, thus leading to a considerable reduction in essay scoring costs. We describe the design and evaluation of the e-rater automatic essay scoring system and discuss some implications of this research for assessment. 1