Gesture, speech, and communication: Exploring gesture style and covert gesture activation

Gesture, speech, and communication: Exploring gesture style and covert gesture activation. Thomas C. Gunter MPI for Human Cognitive and Brain Sciences, Germany In everyday face-to-face conversation, people not only use speech to transfer information but also rely on facial expression, body posture and gestures. In this talk I will discuss whether gestures are perceived as being part of the communication style of a particular person and in how far neuroscience can be used to investigate gesture production. The first part of the talk focusses on the use of iconic gestures. Iconic gestures have a close formal relationship to the semantic content of speech. For instance, a speaker might perform a typing movement with her fingers while saying: “Yesterday I wrote the letter”. Clearly, a listener can extract additional information from these gestures (e.g. we know that the letter was written on a keyboard and not with a pen) and there is no doubt that iconic gestures are communicative and can be integrated online with speech. Note, however, that not every person has a natural habit of using gestures consistently: some people gesture very clearly whereas others are doing a lot of hand waving. The first experiment will show a clear impact of a particular gesturing style on the integration of gestures and speech. The second part of the talk focusses on abstract pointing gestures. In contrast to concrete pointing gestures where the pointing is directed to a physically present target, abstract pointing is directed to ‘empty space’. The parts of space indicated by such gestures get temporally a representational value for the purpose of discourse and can theoretically be used to track concrete and abstract components of the discourse. I will start off by showing that abstract pointing can be used to track references of a conversation. Then, I will show electrophysiological evidence that, while answering a question related to a conversation, there is covert gesture activation preceding the actual utterance of the participant. In other words, whereas no observable abstract pointing activity was present in the behavior/muscles of the participants, there was a clear lateralization of motor cortex activity (related to the abstract pointing gesture of the chosen answer) preceding their utterances. Taken together, the discussed experiments suggest that gesture and speech are highly intertwined streams of information which are taken into account on an individual basis