Examining Rate of Growth in a Computer Adaptive Test of Reading

Background: Universal screening is a critical component within a multi-tiered system of supports because it is the first mechanism by which students are identified as at-risk for failure. Screeners must have adequate psychometric properties to accurately identify students at-risk, however many measures result in moderate accuracy, indicating the presence of false negatives and false positives. These errors may affect the allocation of resources in schools, sometimes causing resources to be utilized unnecessarily and other times withholding services when they are needed. Purpose: This study examined the validity and diagnostic accuracy of a reading CAT using single time points and slopes within year, and longitudinally across one and two years. Methods: Screening and state test data were analyzed for students in third through fifth grade (N = 1696). Validity statistics were calculated using single time points and slopes on the CAT. Diagnostic accuracy analyses included the use of ROC curves to identify the CAT cut score associated with maximized sensitivity and specificity, and the cut score at the 25th and 40th percentile. Results: Correlation results for single screener points and the state test ranged from .60 to .79 (p = .0001). Correlation between slopes and the state test were weak for third and fourth grade (.21 and .14, respectively, p = .0001) and not significant for fifth grade. When ROC curves were used to identify the cut score associated with maximized sensitivity and specificity, overall accuracy was high for single points and moderate for slopes. Accuracy remained high for the cut score at the 25th and 40th percentile, though sensitivity levels were lower for the 25th percentile. Slopes yielded inadequate levels of sensitivity. Conclusion: Results suggest CATs may address issues of inadequate levels of diagnostic accuracy found in the screening literature but additional research is needed in samples with different base rates.Psychological, Health, and Learning Sciences, Department o