Is that a good spoken language dialogue system

This paper steps back from the specific issues involved in evaluating spoken language dialogue systems (SLDSs) to discuss instead the meta-questions of what constitutes a good SLDS and how we can decide if system A is better than system B. It is argued that system goodness is not a property which is amenable to scientific study. Rather, what science can do is unravel the numerous factors which, viewed in isolation, contribute to SLDS quality. At some point, even that exercise stops being part of science and becomes an integral part of manufactoring. 1