Cross-modal representation of spoken and written word meaning in anterodorsal pars triangularis

INTRODUCTION: For years the commonalities underlying the semantic processing of different input-modalities has been studied by means of univariate analysis but nowadays Representation Similarity Analysis (RSA) provides a new opportunity to better understand the nature of such common processes. In two event-related fMRI experiments we searched for regions with activity patterns reflecting a cross-modal semantic similarity effects between written and spoken word modality. METHODS: 18 and 20 healthy subjects participated in the first and second experiment respectively. 24 concrete nouns were used which refer to animals. Based on a feature generation experiment conducted by De Deyne et al. (2008), the pairwise semantic cosine similarity was calculated for each pair of items (semantic cossimilarity matrix). During fMRI subjects performed a property verification task in spoken and written modality. The fMRI data of the first experiment were modelled using a General Linear Model (GLM). Clusters showing a significant main effect of task in the first experiment were used as VOI in the second experiment. The cosine similarity matrix based on fMRI data of the second experiment was generated by calculating the pairwise cosine similarity between every pair of cross-modal trials (cross-modal cossimilarity matrix). Finally we conducted the RSA between the semantic cossimilarity matrix and the cross-modal cossimilarity matrix. RESULTS: The main effect of task (uncorrected p<0.001 combined with a cluster-level corrected p