An Investigation of the Predictive Validity of Kindergarten Dynamic Indicators of Basic Early Literacy Skills (DIBELS) to the Third Grade Pennsylvania System of School Assessment (PSSA)

This study investigated the predictive accuracy of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) administered in kindergarten to reading outcomes on high-stakes standardized testing in third grade on the Pennsylvania System of School Assessment (PSSA). Prediction to DIBELS third grade Spring Oral Reading Fluency (ORF) was also investigated. Participants were 341 students in two cohorts attending a school district in Southeastern Pennsylvania. For Cohort 1 (N = 163), DIBELS data were collected only during kindergarten, whereas for Cohort 2 (N = 178), DIBELS data were collected during kindergarten, first, second, and third grade. As expected, students’ DIBELS scores significantly increased over time during kindergarten across cohorts, and also during first, second, and third grade in Cohort 2. Significant main effects of gender were found on all kindergarten DIBELS measures (Initial Sound Fluency -ISF, Letter Naming Fluency -LNF, Phoneme Segmentation Fluency -PSF, and Nonsense Word Fluency -NWF) in the total sample, with girls outperforming boys on all measures. Effect sizes, however, were small. Predictive correlations indicated that kindergarten Winter LNF, kindergarten Spring LNF, and kindergarten Spring NWF had the largest rs when predicting to the PSSA in third grade (mean r = .49, .51, and .49, respectively, across cohorts). ORF (Cohort 2) was the only DIBELS measure that produced higher correlations than kindergarten Winter LNF. First grade Spring ORF (r = .63) produced nearly as high a correlation as third grade Fall ORF (r = .66). Regression analyses across cohorts revealed that kindergarten DIBELS accounted for 34% of the variance in PSSA scores in third grade. Kindergarten Winter DIBELS explained a significant proportion of variance (14% across cohorts), but Fall did not make a significant contribution when added after Winter measures (1% across cohorts). After adding first grade Spring ORF to the model, 41% out of 47% of the variance was explained; adding in second grade Spring ORF and third grade Fall ORF only explained an additional 6%. Regression analyses predicting to third grade Spring ORF in Cohort 2 revealed that 58% out of 76% of the variance in third grade Spring ORF was explained when first grade Spring ORF was added to the model. The data were also analyzed by dichotomizing scores on the DIBELS and the PSSA into Proficient and Non-Proficient categories, based on DIBELS and Pennsylvania established cutpoints, respectively. Among kindergarten measures, decision statistics revealed relatively higher rates of specificity (mean = 83%) and negative predictive value (mean = 86%) than sensitivity (mean = 42%) and positive predictive value (mean = 36%). Among first grade DIBELS measures (Cohort 2), first grade Spring ORF produced the highest sensitivity (74%), specificity (81%), and odds ratio (11.9). Results of decision statistics and ROC/AUC analyses indicated substantial numbers of false positives and false negatives across all DIBELS subtests. If schools wish to identify students at-risk for reading problems as early as possible, results supported the use of Winter LNF as a screening tool in which students identified as at-risk could either receive necessary reading interventions, or be further evaluated using additional assessment instruments. Due to substantial errors of prediction at the individual level, however, it is necessary for schools to conduct DIBELS testing multiple times throughout the year to avoid the costs of missing students truly at-risk or, conversely, over-identifying those not at-risk. DIBELS benchmarks may be too low for higher SES populations, and too high for lower SES populations. To improve diagnostic accuracy, schools may want to develop their own district-specific norms. Additionally, because the PSSA measures a broader range of reading skills than just reading fluency, the DIBELS may need to be supplemented with measures of reading comprehension for a larger percentage of the variance in PSSA to be predicted