Effects of instruction on semantic and pragmatic judgment tasks

Sentence judgment tasks are used often in linguistics studies. However, there is no consensus on how significant the effect of instruction is in such tasks: some argue that instruction is trivial, while others argue that they affect the way participants respond. In this study, we investigate different keywords used in sentence judgment tasks and determine which keyword best teases apart speakers\u27 response to semantically and pragmatically licit and illicit sentences. We test this in English and Mandarin, exploring the possibility of cross-linguistic variation on how speakers respond to different keywords. Our results show that the common keywords used in semantic and pragmatic judgment tasks such as \u27natural\u27 do distinguish semantic and pragmatic violations for English speakers, but that the common Mandarin translations of these words fail to distinguish between the two types of violations. Our results highlight the need for language- and study-specific norming procedures in sentence judgment tasks