Seeing or Not Seeing Your Conversational Partner: The Influence of Interaction Modality on Prosodic Entrainment

In speech entrainment research, a less investigated yet crucial aspect for modelling multimodal interactive dialogue systems is the influence of inter-action modality, i.e. whether conversational partners who are visible to each other would entrain their speech more with respect to when eye contact is inhibited, or not. In our study, we compared prosodic adaptation behaviour (convergence and synchrony) of the same speaker pairs involved in collaborative game sessions under two conditions: audiovisual vs audio-only interaction. Results provide a complex picture, with a tendency to vocal entrainment enhancement when the speech/audio channel is the only one available