Colored-Speech Synaesthesia Is Triggered by Multisensory, Not Unisensory

ABSTRACT—Although it is estimated that as many as 4 % of people experience some form of enhanced cross talk be-tween (or within) the senses, known as synaesthesia, very little is understood about the level of information pro-cessing required to induce a synaesthetic experience. In work presented here, we used a well-known multisensory illusion called the McGurk effect to show that synaesthesia is driven by late, perceptual processing, rather than early, unisensory processing. Specifically, we tested 9 linguistic-color synaesthetes and found that the colors induced by spoken words are related to what is perceived (i.e., the illusory combination of audio and visual inputs) and not to the auditory component alone. Our findings indicate that color-speech synaesthesia is triggered only when a signifi-cant amount of information processing has occurred and that early sensory activation is not directly linked to the synaesthetic experience. Synaesthesia occurs when a stimulus triggers an anomalous percept in the same or different modality as the normal percept. Reported synaesthetic experiences have ranged from tasting words (Ward & Simner, 2003) and sounds (Beeli, Esslen, & Jäncke, 2005) to seeing calendar units (Smilek, Callejas, Dixon, & Merikle, 2006), but by far the most studied form of syn-aesthesia is induction of color by letters, words, or digits (Rich & Mattingley, 2002). This latter type of synaesthesia is diversely referred to as colored speech (Baron-Cohen, Harrison, Gold-stein, & Wyke, 1993), colored hearing (Marks, 1975), or lin-guistic-color (Simner, Glover, & Mowat, 2006), lexical-colo