A Systematic Investigation of Program Differentiation Within a Kindergarten Reading Intervention Study: The Importance of Accounting for Implementation Across Conditions

This study examined dimensions of treatment integrity, specifically program differentiation, within the context of a kindergarten reading intervention study. The study explored the relationship between program differentiation and student outcomes. The study was conducted within the context of Project Early Reading Intervention (ERI), a four-year Institute of Educational Sciences funded study designed to test the efficacy of a published reading intervention with kindergarten at-risk readers. ^ Participants included 46 kindergarten interventionists and 151 kindergarten students. As part of Project ERI, intervention groups consisted of 3-5 students. Groups were assigned to receive either the Early Reading Intervention curriculum or school-designed intervention. All students received 30 minutes of intervention five days per week. ERI study protocol included the videotaping of each interventionist on three occasions over the course of the intervention. Within the present study, these video-recorded lessons were coded using the researcher-developed Reading Intervention Observation Tool (RIOT). The RIOT was designed to capture data on instructional content and interventionist behaviors across the entirety of the lesson. Momentary time sampling was used to achieve this objective. ^ First, t-tests were performed in order to determine if interventionists in the ERI condition had adhered to ERI protocol. These interventionists were compared to an exemplary interventionist. In order to analyze differences in instructional content and instructional behaviors between the ERI and school designed intervention conditions, t-tests were performed on RIOT data. Tests analyzed both instructional content and interventionist behaviors. Finally, attempts were made to relate RIOT variables to student outcomes using Hierarchical Linear Modeling (HLM). ^ Results suggested that ERI interventionists did not satisfactorily adhere to the ERI curriculum. ERI as implemented and school designed intervention were statistically comparable across many of the dimensions measured by the RIOT. Thus, the ERI and school designed interventions were more similar than expected in terms of instructional content and interventionist behavior. Finally, Hierarchical Liner Models attempting to relate key RIOT variables to student outcomes were not statistically significant. Implications and limitations of these findings are discussed.