Multimodale dynamiek in een getolkt gesprek

Recent years have witnessed a growing interest in multimodal features of spoken language (Müller et al. 2013), that crucially involve the interaction of the auditory (speech), visual gaze) and kinesthetic (gesture, posture) modalities in construing and communicating meaning. Especially eye-gaze patterns in (semi-)naturalistic settings are receiving increasing attention in multimodal interaction research. The present study pursues this recent line of research by inquiring into the role of eye-gaze in correlation with other (non-)verbal signals in the specific institutionalized setting of interpreter-mediated dialogue. Studies have shown that synchronization and turn transfer in interpreted dialogues are collaboratively coordinated through verbal and non-verbal means (Wadensjö 1998, Bot 2005, Mason 2012). Especially gaze direction has an important function in signaling conversational attention and facilitating turn taking in interpreted dialogues (Lang 1978, Wadensjö 2001, Bot 2005, Mason 2012). In this study we aim at an in-depth analysis of the gaze patterns of the participants in interpreted dialogues in relation to speech, gesture and posture and how they function in the constitution of dialogue management. We collected a corpus of semi-authentic interpreter-mediated interactions by using mobile eye-tracking technology (Gullberg & Kita 2009, Jokinen 2010, Brône & Oben 2013, Oben & Brône fc.). The data annotation and statistical processing of the gaze data was largely based on the workflow developed in Brône et al. (2013) for face-to-face dialogues. The analysis of gaze patterns is indicative of interpreter’s regulatory role in this type of dialogue. Moreover, the study provides insights in the processes of turn-taking, footing (e.g. speaker selection) information management and grounding (or the process of establishing common ground) in interpreted interaction.status: publishe