Building a semantically transparent corpus for the generation of referring expressions

This paper discusses the construction ofa corpus for the evaluation of algorithmsthat generate referring expressions. It isargued that such an evaluation task requiresa semantically transparent corpus,and controlled experiments are the bestway to create such a resource. We addressa number of issues that have arisen in anongoing evaluation study, among which isthe problem of judging the output of GREalgorithms against a human gold standard