Situated interaction with a virtual human - perception, action, and cognition

Leßmann N, Kopp S, Wachsmuth I. Situated interaction with a virtual human - perception, action, and cognition. In: Rickheit G, Wachsmuth I, eds. Situated Communication. Berlin: Mouton de Gruyter; 2006: 287-323.In Virtual Reality environments, real humans can meet virtual humans to collaborate on tasks. The agent Max is such a virtual human, providing the human user with a face-to-face collaboration partner in the SFB 360 construction tasks. This paper describes how Max can assist by combining manipulative capabilities for assembly actions with conversational capabilities for mixed-initiative dialogue. During the interaction, Max employs speech, gaze, facial expression, and gesture and is able to initiate assembly actions. We present the underlying model of Max’s competences for managing situated interactions, and we show how the required faculties of perception, action, and cognition are realized and connected in his architecture