A study of how Pennsylvania superintendents use data-driven decision-making and Standard and Poor\u27s School Evaluation Services to understand and improve district performance

In October 2001, Standard and Poor\u27s School Evaluation Services (S&P SES) began displaying and analyzing district performance data for every Pennsylvania school system. S&P SES is a second-generation external accountability system that goes beyond listing traditional first-generation accountability components, such as test scores and drop-out rates, to include education-sector financial analysis linking school districts\u27 academic results with program spending. ^ This research study examines how S&P SES is perceived by Pennsylvania superintendents. A survey designed specifically for this study asked each Pennsylvania superintendent to identify the research-based data-driven decision-making practices that are essential components of the district\u27s internal accountability process. Each superintendent also was asked to rate how helpful the resources of S&P SES\u27 were to them while they were using the data-driven decision-making practices identified as essential. Superintendents also rated the accuracy and comprehensive nature of the data supplied to S&P SES, and they listed S&P SES\u27 strengths and limitations. The survey research was followed by telephone interviews of ten volunteer superintendents. The data revealed that 23 of the 38 researched-based data-driven decision-making practices were considered essential by superintendents, but only 9 of those 23 were helped by the resources of S&P SES. 62% of the surveyed superintendents were confident in the accuracy of the data, and 53% were confident in how comprehensive the data was. Short answer responses revealed that the superintendents found the ability to compare districts, the Performance Cost Index (PCI) information, the financial comparisons, and the ease of use of S&P SES to be strengths. Limitations cited were the outdated nature of some data, PCI use without reference to unique characteristics of districts, the misleading nature of some comparisons, and the difficulty interpreting the data due to confusing operational definitions. In addition, the short answers and the interviews revealed a perception, which some consider to be wide-spread, that the implementation of S&P SES has been compromised because of its association with a negative political agenda.