Emotions by Ear and by Eye

© Springer Science+Business Media New York 2013. Multisensory integration must stand out among the fields of research that have witnessed one of the most impressive explosions of interest this last decade, at least as measured by published papers and meetings. From a highly specialized niche occupation multisensory research has become a mainstream scientific interest in a very short time span. One of these new areas of multisensory research is emotion. Since our first exploration of this phenomenon [de Gelder Böcker, Tuomainen, Hensen, & Vroomen Neuroscience Letters 260(2):133-136, 1999], a number of studies have appeared and they have used a wide variety of behavioral, neuropsychological, and neuroimaging methods. The goal of this chapter is fourfold. First, we review the research on audiovisual perception of emotional signals from the face and the voice. In the next section we discuss some outstanding methodological and theoretical issues followed by a report and comment on integrating emotional information provided by the voice and whole body expressions. We also include some recent work on music. Finally, we discuss findings about abnormal affective audiovisual integration in schizophrenia and in autism.edition: 1ststatus: publishe