Evaluating algorithms for the Generation of Referring Expressions using a balanced corpus

Despite being the focus of intensive research, evaluationof algorithms that generate referring expressionsis still in its infancy. We describe a corpusbasedevaluation methodology, applied to a numberof classic algorithms in this area. The methodologyfocuses on balance and semantic transparency toenable comparison of human and algorithmic output.Although the Incremental Algorithm emergesas the best match, we found that its dependency onmanually-set parameters makes its performance difficultto predict