Grammaticality judgment task: Reliability and scope

© 2004 AsiaTEFL.org. All rights reserved. Although extensively used, concerns have been expressed about the reliability and generalizability of grammaticality judgment tasks (GJTs; Alanazi, 2015). It has been argued that learners might guess grammaticality based on feel (Bialystock, 1979) and that the GJT-based results ignore grammatical complexity (Ellis, 1991). While several studies have attempted to validate the results of a GJT against other tasks, the tasks used in these studies required language production (e.g., Leow, 1996; Mandell, 1999), which is different from error identification and correction. The current study used an editing task (ET) to validate learners’ performance on a GJT. An editing task, like a GJT, requires identification of grammatical inaccuracies. Besides, it situates errors in a meaningful context and offers opportunities for multiple corrections of the same error. Overall, 311 participants took part in the study. A paired sample t-test showed a significant difference (t = 24.10, p = .00, d = 1.91) between the two tasks. Fifteen percent of the data (45 participants) was further inspected to have a better understanding of learners’ error correction-pattern on the editing task. The results revealed that about 20% of the errors identified were wrongly corrected, which exposes limitations of the GJTs that only require judging the well-formedness of a construction. Moreover, performance on the editing task revealed a greater level of diversity in learners’ responses. On average, eight of the total twelve grammatical features were corrected in 5.87 (SD = 2.47) different ways. The variety of responses on the ET reveals the dynamic nature of learners’ interlanguage that allows for multiple ways for correction. The ET, as compared to the GJT, provides an ecologically more valid way to assess L2 learners’ grammatical proficiency