Are gesture and prosodic prominences always coordinated?: evidence from perception and production

Comunicació presentada a: Speech Prosody 2014, celebrada del 20 al 23 de maig de 2014 a Dublin, Irlanda.This study explores the temporal coordination between gesture and speech by addressing two main questions: (1) Are speakers sensitive to the misalignment between gesture prominence and prosodic prominence? (2) Is this sensitivity modulated by the semantic information conveyed by gesture and speech modalities in production? Experiment 1 tested question (1) and Experiment 2 tested question (2). Results from Experiment 1 revealed that the combinations in which prominences were misaligned were less acceptable than combinations with aligned prominences, and that the metrical pattern of the target word had an effect on the speakers’ sensitivity: unsynchronized trochees (with the gesture prominence at the post-tonic syllable) were frequently accepted, while unsynchronized iambs (with the gesture prominence at the pre-tonic syllable) were rejected. Results from Experiment 2 revealed that when the pointing gesture adds information to speech, i.e. it is supplementary to speech, the prominences are frequently misaligned (with gesture occurring after the speech), as if two different speech acts were produced. These findings suggest that the semantic content of gesture-speech combinations might influence the speakers’ sensitivity of the misalignment between prosodic and gesture prominences.This research has been funded by grants FFI2012-31995, PSI-2011-25376, 2009SGR-701, and by the RECERCAIXA 2012 grant “Els precursors del llenguatge: una guia TIC per a pares i educadors”