Producing Believable Robot Gaze When Comprehending Visually Situated Dialogue

The paper presents an implemented approach to produc-ing robot gaze during comprehending visually situated di-alogue. The approach is based on an incremental model for processing situated dialogue. In this model, utterance interpretations are build step-by-step, in a ”left-to-right” fashion. At each step, grammatical and dialogue-level in-formation is combined with information about the visually situated context. As a consequence, utterance processing can be guided so as to construct only situationally appro-priate interpretations. Furthermore, at each step a set of visual referents is determined, to which the unfolding ut-terance meaning is currently making reference. In the ap-proach, this information is used to drive robot gaze, letting the robot change its fixation onto the most recent visual referent. The underlying assumption is that gaze behavior helps to establish joint attention (”common ground”) in a dialogue, if there is congruency between where the robot is looking, and what the (intended) visual referent is. The paper reports on a pilot study in which this assumption is studied. The results show statistically significant interac-tions between congruence, believability, and appropriate-ness of referring expression