Format and Training Effects on Rating Accuracy and Rater Errors

Examined the effects of rater training and rating format on halo and accuracy in performance ratings. A training program previously found to reduce the number of rating errors was adapted for use here, and 5 rating formats that have shown promise in helping to reduce rating errors were developed for 2 jobs. As in previous research, 5- to 9-min videotapes of actors performing on these 2 jobs according to scripts depicting realistic levels of effectiveness on the various performance dimensions provided the ratees for Ss to evaluate. 123 college students were assigned randomly to 1 of 10 cells defined by 5 format and 2 training treatments (trained vs untrained), and raters evaluated the effectiveness of the videotaped ratees\u27 performances. Results show that training significantly reduced halo, but this finding was uneven across the 2 jobs. Training did not improve accuracy. Significant format effects were found for both halo and accuracy, but a significant job by format effect on accuracy suggested that no one format is consistently best across situations. Possible training strategies to enhance rating reliability and accuracy are discussed