Language modality and temporal structure impact processing: Sign and speech have different windows of integration

Available online 27 August 2021.Language comprehension depends on the ability to temporally process the periodic structure of the language signal. In this study we investigate temporal processing of Spanish Sign Language (LSE), isolating the specific contribution of signal modality through a comparison with spoken Spanish and the contribution of linguistic status by comparing language with a non-linguistic temporally structured visual signal. Twenty-three highly proficient hearing users of LSE performed an intelligibility task with these three types of materials, manipulated with different levels of temporal distortion. The results show that the distortion differently affects the intelligibility of these signals. Spanish is characterized by a threshold of temporal distortion, beyond which intelligibility rapidly decreases and is almost completely lost. Conversely, in LSE and the visual non-linguistic task greater temporal distortion led to a gradual and constant reduction in intelligibility with no clear threshold. LSE is more resilient to temporal manipulation compared to the visual non-linguistic signal: participants’ performance never drops below 50% even with the most severe distortion. Overall, these findings suggest that the temporal processing of language arises from the complex interaction between the properties of the sensory system and the special characteristics of the language signal.The authors acknowledge financial support from the Basque Government through the BERC 2018-2021 program, from the Spanish Ministry of Science and Innovation (Agencia Estatal de Investigaci´on, AEI), through grants (PSI-2016-76435-P to B.C.; and RTI2018-093547- B-I00 to M.C.), through Juan de la Cierva Fellowship (FJCI-2017- 31806 to B.C.) and from “La Caixa” Foundation and the European Union’s Horizon 2020 research and innovation programme under the Marie-Skłodowska-Curie grant agreement no. 713673 (personal fellowship code LCF/BQ/DI18/11660008 to C.L.R.). This research was founded by the Severo Ochoa Center of Excellence grant (CEX2020- 001010-S)