Expecting the expected : Semantic incongruities are judged differently depending on the source during sentence judgment tasks

How well can people identify semantic incongruities, which are often used in linguistic research and experimental psychology (e.g., Clifton, Staub, & Rayner, 2007)? Acceptability judgment tasks measure how natural a sentence sounds (e.g., Dąbrowska, 2010), and are often used with sentences with or without incongruities, but can these judgments be affected by expectation and bias? Two experiments were performed, where participants read either semantically congruent or incongruent sentences, rating each sentence immediately after reading it. Sentences were given without explicit framing in Experiment 1, but were said to be from either “well respected” or “less respected” news sources in Experiment 2. We created the semantic incongruities by replacing a word with its antonymic partner, such as ‘soft’ instead of ‘hard’, ‘good’ instead of ‘bad’, and ‘large’ instead of ‘small’. Previous research suggest that participants have difficulties differentiating between grammatical and pragmatic violations, between grammaticality and acceptability judgments (Ariel, 2010, p. 77). We therefore used separate syntactic and semantic acceptability judgment tasks to investigate how connected these two judgments would be. Two additional judgment tasks allowed the participants to state their interest in reading the rest of the article based on the form and the topic of the sentence, resulting in form-based and topic-based interest ratings. In Experiment 1, no significant differences were found for any rating. In Experiment 2, syntactic acceptability, semantic acceptability, and form-based interest ratings decreased significantly for sentences from less respected sources compared to well-respected ones. Specifically, the difference in semantic acceptability ratings was significant for semantic incongruities, showing that semantic incongruities were correctly identified for less respected sources only. Surprisingly, topic-based interest ratings increased significantly for less respected sources compared to well-respected ones. The results suggest that people have a negative bias toward the form in less respected sources, considering them more prone to mistakes and thus more likely to justify or overlook errors in well-respected sources. Thus, people might not always be accurate when identifying semantic incongruities in judgment tasks, as they only correctly identified them when expecting less respected sources in this study. The results also suggest that people have a negative bias toward the topics in well-respected sources, possibly considering them boring or dry, and therefore are less likely to want to read them. In this study, these biases arose from expectations, not manipulations, showing that judgment tasks are affected by expectation and bias. ReferencesAriel, M. (2010). Defining pragmatics. Cambridge: Cambridge University Press.Clifton, C., Staub, A., & Rayner, K. (2007). Eye movements in reading words and sentences. In R. P. G. Van Gompel, M. H. Fischer, W. S. Murray, & R. L. Hill (Eds.), Eye movements: A window on mind and brain (pp. 341–371). Elsevier Ltd. http://doi.org/10.1016/b978-008044980-7/50017-3Dąbrowska, E. (2010). Naive v. expert intuitions: An empirical study of acceptability judgments. The Linguistic Review, 27, 1–23. http://doi.org/10.1515/tlir.2010.00