When emotional prosody and semantics interact in time: ERP evidence

Emotional encoding is central to human communication. To this end, it is not only important to understand the verbal content of the message (i.e. its semantic), but also to understand the emotional prosody that accompanies the message. So far it is yet not fully understood in which way and at which point in time emotional prosody and (emotional) semantics interact at the sentence level. However, previous evidence (Kotz et al., 2001) showed that the time course of emotional prosody and semantics differ. In order to investigate the interaction of the two channels we carried out two ERP experiments in which we tried to isolate the emotional prosody channel from the semantic content channel using a cross-splicing method. The ERP evidence shows that violations of the emotional prosodic contour (with neutral semantic content) elicit a positivity whereas combined emotional prosodic and semantic content violations elicit a negativity, comparable to the N400 component. This pattern evolves irrespective of the task used, i.e. the same pattern emerges for an “implicit” (probe detection) task as well as for an “explicit” (emotional prosody categorization) task. We conclude that prosodic information can be processed separately from semantic information, and that each channel (prosody and semantic) contributes differently to the perception of emotional speech. Furthermore, it appears that semantic information overrides prosody when the two channels interact in time when the emotional prosodic contour agrees with the semantic content of a sentence