Use of Rasch Rating Scale Modeling to Develop and Validate a Measure of District-Level Characteristics and Practices Identified to Improve Instruction and Increase Student Achievement

Increasing qualitative evidence in the literature supports specific characteristics and practices, presented from a distributed leadership perspective, to be prevalent in school districts that demonstrate significant increases in student achievement. Quantitative evidence linking these identified district-level characteristics and practices to increased student achievement, however, is lacking. Lack of quantitative evidence appears due to want of an acceptable, psychometrically sound instrument. The purpose of this study was to create an instrument, the District Leadership for Effective Schools Scales (DLESS), to address this void. The DLESS is a measure of district-level characteristics and practices identified to improve instruction and increase student achievement. The activities suggested by Wolfe and Smith (2007a, 2007b) to facilitate the documentation of evidence relating to the development and validation of measures using Rasch modeling served as the outline for the DLESS’s development. The Ohio’s Leadership Developmental Framework served as the conceptual framework, serving as progenitor of the DLESS’s test specifications. For the validation study, the DLESS was completed by 277 superintendents of traditional K-12 public schools in Ohio. The sample demonstrated excellent representation of school districts in Ohio. Qualtrics on-line survey software was used to recruit participants, distribute the DLESS, and collect responses. Responses were analyzed using the Rasch measurement procedures contained in WINSTEPS Version 3.73. Linacre’s guidelines for optimizing rating scale effectiveness were used to direct the analyses. Pearson correlations of the DLESS measure with a measure of district-level student achievement were calculated using SPSS. Results indicate the DLESS adequately meets the Rasch specifications and is a well functioning scale with adequately functioning rating scale categories. Separation indices for persons and items indicate an excellent degree of variability between persons and items making it appropriate for examining districts within a broad range of the construct. The reliability for persons (.93) and items (.98) is excellent. A weak though statistically significant positive correlation was found between the DLESS and a measure of district-level student achievement. The strength of the correlation was found to increase over time