An acoustical and cognitive approach to the semiotics of sound objects

International audienceIn this work we investigated the relation between semantic processing in language and semiotic processing of “sound objects”. More precisely we designed an experiment to test whether there can be semantic priming across linguistic and sound object domains. At this aim we selected sounds based on an acousmatic approach, meaning that the source that produced the sounds could not be recognized, giving the listener the possibility to experience the sound for its own sake disconnected from its original context. For this purpose, we built a rather large corpus of short sounds and asked 10 subjects to write one or more words that they believed to be related to each sound. Then we selected the most consistent 45 sound/word related pairs. We used the selected material in an experiment wherein words were shortly presented on a computer screen and were followed by a sound that could be related or non-related. We asked subjects to decide as quickly as possible whether the word/sound pair was related or not. We also recorded EEG signal from 32 scalp electrodes. Preliminary results indicate a negative component time-locked to sounds that resembles the N400 component described for semantic processing of words. Moreover, sound processing seem to be strongly influenced by the semantic role (related/non related) of preceding words. Finally, precise acoustical analyses of sounds give a hint on how acoustical properties relate to meaning. This study allows to better understand the cognitive processing of complex sounds and its relation to linguistic semantic processing