8. Limitations of Using Student-Achievement Data for Career-Ladder Promotions and Merit-Pay Decisions

A study of U.S. school districts conducted 70 years ago reported that 48% of the districts sampled used merit pay (Evendon, 1918). Since then, the quantity as well as quality of teacher-compensation systems has fluctuated markedly (for details, see Cohen & Murnane, 1985; Murnane & Cohen, 1986; Porwoll, 1979). At present, 29 states are implementing large-scale teacher-incentive programs (a.k.a. career ladder, merit pay, pay for performance), funding local plans, piloting testing models, or using state board of education or legislative mandates to develop programs for teachers and administrators (Southern Regional Education Board, 1986) The status of these programs is summarized in Table 8.1. Teacher performance is at the core of all of the programs in operation or those being considered. Determining who will receive the pay bonuses, which typically range from $1,000 to $3,000 per year, or be promoted up the career-ladder hinges on the methods used to evaluate teacher performance. The current trend in measurement procedures is to deemphasize supervisory ratings by the building principal and instead to emphasize peer evaluation, classroom observation, student-achievement outcomes, and questionnaire data from principals, teachers, and students (for details, see Southern Regional Education Board, 1986). Use of Student·Achievement Data One particular procedure that seems to be gaining acceptance increasingly by legislators and the professionals who are designing the programs is the use of student-achievement data (d. Robinson, 1983; 1984). These data provide information different from the other measurement tools previously noted. Where classroom observation and ratings by principals, teachers, and students measure a teacher\u27s behavior on the job, student achievement relates to the outcomes of that behavior. That is, the former methods are direct measures of teacher performance; the latter is an indirect measure. Student outcomes are perceived as evidence of a teacher\u27s effectiveness. Because superior teacher performance is the criterion in teacher-incentive programs, the psychometric issue becomes how best to measure that performance-use direct measures, indirect measures, or a combination of both. Teacher-incentive programs that rely on student-achievement gains have been referred to as new style merit pay (Bacharach, Lipsky, & Shedd, 1984), as opposed to old style merit pay, which bases teacher pay bonuses on principals\u27 evaluations. In 1983, a national survey of merit-pay programs reported that nine school districts in seven states (Arizona, New Hampshire, North Carolina, Oklahoma, South Dakota, Texas, Utah) used student-test scores as evaluative criteria in determining merit pay for classroom teachers (Calhoun & Protheroe, 1983). In all but two of the districts (Dallas and Houston) student achievement served as the only evidence of teacher performance. Today student achievement is a criterion of teacher performance in one third of all statewide teacher incentive/school incentive/career ladder programs. Those programs have been fully implemented in four states (Florida, South Carolina, Tennessee, Utah), are at the pilot stage in four states (Arizona, Kentucky, Maine, South Carolina), and are under development in three states (Alabama, Florida, Georgia). A school incentive program based on student achievement is also under consideration in Alaska, and several career-ladder or merit-pay programs based on student performance have been implemented by local districts (e.g ., Campbell County and Danville, Virginia)