Linguistic features as possible sources for inequivalence of mathematics PISA tasks

When mathematics tasks are translated to different languages, there is a risk that the different language versions are not equivalent and display differential item functioning (DIF). In this study, we aimed to identify possible sources of DIF. We investigated whether differences in some linguistic features are related to DIF between the English (USA), German, and Swedish versions of mathematics tasks of the PISA 2012 assessment. The linguistic features chosen in this study are grammatical person, voice (active/passive), and sentence structure. We analyzed the three different language versions of 83 mathematics PISA tasks in three steps. First, we calculated the amount of differences in the three linguistic features between the language versions. Then, we calculated DIF, using the Mantel-Haenszel procedure pairwise for two language versions at a time. Finally, we searched for correlations between the amount of linguistic differences and DIF between the versions. The analysis showed that differences in linguistic features occurred between the language versions – differences in voice were most common – and that several items displayed intermediate or large level of DIF. Still, there were no statistical significant correlations between differences in linguistic features and DIF between the language versions, that is, there must be other sources of DIF