The Impact of Gatekeepers on Evaluation Use

By examining the development of a local school reform that was meant to be teacher-driven, this multiple case study explored the limits on evaluation use in a hierarchical setting in which multiple audiences exist. This study was a comparative analysis of the actions and evaluation philosophies of two district administrators responsible for overseeing the same reform at different times. A mixed method approach was employed to analyze secondary data to answer the overarching research question of how evaluation gatekeepers facilitated the use of evaluation findings. Qualitative data suggested that a myriad of factors influenced the actions taken by each of the directors, whom were identified as the main gatekeepers of evaluation information. The director administering the grant in its last two years appeared to be the most committed to the evaluation and most influential in driving use. This was supported by qualitative data on his administrative style, how he navigated the school bureaucracy, his personal beliefs of the SLC initiative, how he monitored and/or prescribed evaluation use, and the amount of time he dedicated to professional development on evaluation use. Additional factors influencing use were also identified. Survey data suggested that school organizational structure influenced utilization by potential users. In general, leadership position at the school level influenced evaluation attitudes and awareness. Principals and SLC leaders were the most aware of the evaluation and its findings. While teachers expected principals to be the main users of evaluation findings, principals themselves did not necessarily hold the same expectation of themselves. Use was reported primarily as leading to direct changes to the SLC programs and, secondly, for planning purposes. There were also some minor references to the use of the evaluation findings for generating conversations/discussions. The findings generated from this multiple case study provide lessons that need further in-depth exploration. The factors identified in this study as the most influential to evaluation use can be studied either individually in simulations or as a group case study. Doing so would bring greater awareness to practitioners about how to encourage evaluation use and, perhaps, greater appreciation for evaluations to stakeholders