Psychometric Characteristics of Academic Language Discourse Analysis Tools

Academic language plays a key role in students’ educational success, yet its development in primary grades is poorly understood and often neglected (Snow & Uccelli, 2008). Academic language skills may enhance overall academic performance if targeted early and intensively. However, current methods of assessment are not sufficient to understanding the construct well enough to develop evidence-based intervention strategies. This investigation examined the psychometric properties of two discourse analysis tools designed to directly measure students’ comprehension and production of academic language. Academic language samples (n = 7,887) from a previous cohort-design study (n = 1,040; Kindergarten through third grade participants) were scored using the Narrative Language Measure (NLM) Flowchart and the Expository Language Measure (ELM) Flowchart. A confirmatory factor analysis was used to test two-factor models for both flowcharts. The total scores and subscale scores of the NLM Flowchart demonstrated moderate to strong interrater reliability, moderate convergent validity, and approximate fit with the proposed model (generation χ²(46) = 743.85, p \u3c .001, SRMR = .06, RMSEA = .08, CFI = .88, and TLI = .86; retell χ²(46) = 784.80, p \u3c .001, SRMR = .05, RMSEA = .09, CFI = .91, and TLI = .90). One subscale (i.e., Narrative Structure) showed adequate internal consistency via Cronbach’s alpha. This study found mixed evidence of interrater reliability for the ELM Flowchart, with weak agreement on one subscale (i.e., Passage Structure) and substantial to strong agreement on the other (i.e., Language Complexity). The ELM Flowchart demonstrated moderate convergent validity, but neither subscale reached acceptable levels of internal consistency via Cronbach’s alpha. The appropriateness of using reflective indicator tools to evaluate constructs that may be better suited to a formative model is discussed. Other implications of the findings also are discussed