Examining the Relationship Between a Universal Screening Measure and a State Reading Assessment

Many adolescents struggle with reading comprehension, despite an emphasis on reading instruction over recent decades. Evidence suggests that informational text is particularly challenging for students. To implement assessments within a multi-tiered framework, schools must have psychometrically adequate tools. Universal screening data can be useful when identifying at-risk students in need of intervention. Using data of 473 students in Grades 6 through 8 from two Western Maine middle schools, this study examined the relationship between the Northwest Evaluation Association’s Measures of Academic Progress in Reading (NWEA MAP-R) and Maine’s eMPowerME English Language Arts/Literacy (ELA/L) test by Measured Progress. Logistic regression models were statistically significant with MAP-R scores explaining 54.6% and 58.2% of the variance in proficiency on the eMPowerME ELA/L and correctly classifying 83% and 80% of cases for Grades 6-7 and 7-8. MAP-R had the best balance of sensitivity and specificity in Grade 6-7. Gender, socioeconomic status, and disability status were used to determine if value was added by combining MAP-R scores with student characteristics. The added demographic variables were less robust predictors of reading achievement when combined with MAP-R scores compared to MAP-R scores alone. This investigation also determined the minimum MAP-R scores needed in spring of Grade 6 and 7 that demonstrated a student was on track to meet proficiency standards on eMPowerME one year later. Both grade-level cohorts MAP-R scores resulted in high AUC values; however, diagnostic accuracy was below the acceptable level recommended for a screener when using NWEA-provided cut-scores. By using locally derived cut-scores, the diagnostic accuracy was improved by maximizing sensitivity and specificity to an acceptable level. A secondary purpose of this study was to determine whether differences existed between subtests scores and to examine the unique contribution of MAP-R subtests to the eMPowerME. Students performed better on the MAP-R vocabulary acquisition and use (VAU) subtest compared to either the literary or informational text subtests. The VAU subtest, however, had the weakest correlations with eMPowerME. There was not a statistically significant difference between literary and informational text scores. The MAP-R literary text subtest accounted for the highest degree of variance in eMPowerME ELA/L scores