Shrinkage of value-added estimates and characteristics of students with hard-to-predict achievement levels. Mathematica Policy Research

It is common in the implementation of teacher accountability systems to use a procedure known as Empirical Bayes shrinkage to adjust the teacher value-added estimates by their level of precision. Because value-added estimates based on fewer students and students with “hard-to-predict ” achievement will be less precise, the procedure could have differential impacts on the probability that the teachers of fewer students or students with hard-to-predict achievement will be assigned consequences. This paper investigates how shrinkage affects the value-added estimates of teachers of hard-to-predict students, focusing on the context of threshold-based accountability systems. We found that the achievement of particular groups of students—students with low prior achievement and who receive free lunch—is harder to predict. Teachers of these types of students tend to have less precise value-added estimates, and shrinkage increases their estimates ’ precision. Shrinkage also reduces the absolute value of the value-added estimates of teachers of hard-to-predict students. However, in our data, we found that shrinkage has no statistically significant effect on the relative probability that teachers of hard-to-predict students receive consequences