Teacher Judgments of Student Writing: Examining the Accuracy of Ratings for the Identification of At-Risk Students

This dissertation examined the accuracy of teacher judgments of student writing as compared to two established measures of written language. Previous research in the area of teacher judgments has shown that teacher ratings have moderately high correlations with alternate measures of student performance across academic areas, including reading and math, demonstrating a positive relationship between ratings and performance. However, research which has examined the hit-rates, or individual rates of agreement between teacher judgments and alternate measures, has been less positive, showing that teachers may have difficulty predicting the performance of individual students. Furthermore, very few studies have examined the accuracy of teacher judgments in the area of written language. Twelve teachers and their students (n=226) in third and fourth grades participated in the current study. Teachers completed ratings of their students\u27 general writing skills as well as their specific skills. Students completed the TOWL-4 and two CBM writing prompts through class-wide administration. Hit- and miss-rates were calculated, and correlations were computed between general teacher ratings and overall performance as well as between specific ratings of skills and corresponding measures. Results indicated that teachers had difficulty identifying students at risk for writing difficulties as measured by hit-rates, with a rate of agreement with the TOWL-4 of 76%, and a rate of agreement with CBM measures of 65%. Correlational data between teacher ratings of student writing and performance on the alternate measures indicated moderately high correlations between teacher judgments and student performance on the TOWL-4, and moderate correlations between teacher judgments and performance on CBM measures. Results are discussed in terms of implications for practice and limitations of comparisons between multiple measures