Semantic Congruence Across Sensory Modalities

Multisensory processing often has facilitation and/or interference effects (Chen & Spence, 2010; Thomas et al., 2017) yet the mechanisms remain unclear. This experiment used a Stroop-like task (Stroop, 1935) to examine how congruent, incongruent, and irrelevant information presented in one sensory modality affects processing and responding in a different modality (see Figure 1 for trial types). Participants were simultaneously presented with pictures and sounds of animals or vehicles. In separate blocks they had to respond to either the visual or auditory stimulus, while ignoring the other modality. Results suggest that visual stimuli have a larger effect on auditory responding than vice versa. Incongruent visual stimuli had a larger effect on accuracy while congruent visual stimuli had a larger impact on response time. We will present additional data at the poster focusing on factors that may affect congruency effects including individual differences and types of auditory stimuli