Fairness in teacher evaluation: validity of personal evaluations based on an classroom observation instrument measuring sequential development of teaching skill

Interest in teacher evaluation is mainly motivated by a desire for selection. Given this motivation, it is unrealistic to think that an evaluation instrument will never result in others making summative decisions about teachers’ careers. This study applies the validity argument approach (Kane, 2006) to investigate the validity of personal evaluations obtained with a classroom observation instrument that presupposes sequential development in teaching skills. The study seeks support for two claims; (1) classroom observations of teaching acts show sequential ordering which may reflect sequential stages in teacher development; (2) implementation of this instrument within a pre-planned evaluation method, which prescribes three classroom observations by three different observers, can protect teachers against and inform teachers about bias in their personal evaluations. Using a cross-validation method, it is empirically shown that 26 of the 32 initial items fit all three Rasch assumptions. Results further indicate that most (i.e., 84.2%) of the personal evaluations have moderate fit with the theoretical sequential ordering, but that few (i.e., 36.0%) evaluations show good fit. Furthermore, increasing the number of classroom observations was found to considerably increase the validity of each personal evaluation