THE EFFECT OF ABILITY AND MOTIVATION ON THE ACCURACY OF PERFORMANCE EVALUATION

This dissertation examines the effect of the ability and motivation changes associated with rater training on the accuracy of performance evaluation. Previous studies of rater training have produced equivocal results for two reasons: (1) The majority of these studies use psychometric error as the dependent variable; recent studies indicate little relationship between such rater errors and rating accuracy. (2) None of these studies clearly identify the intended impact of training: motivation, ability or some combination thereof. This study attempts to clearly identify the contribution of ability and motivation to rating accuracy. The use of a 2 x 2 factorial design permits the examination of the effect on rating accuracy of two levels of ability (training vs no training), two levels of motivation (pay vs no pay), and their interaction. One hundred eighteen subjects were randomly assigned to one of four groups. The experimental task involved evaluating the effectiveness of the videotaped performances of six job recruiting interviewers. The dependent variable was the accuracy of subjects\u27 evaluations. None of the variables, ability, motivation or their interaction, significantly affected the accuracy of ratings. However, the data provide several insights for studies of rater training or performance evaluation: (1) Difference scores may be superior to differential accuracy as a measure of rater accuracy. (2) The focus of rater training should be ratees rather than rating dimensions. (3) The measurement of accuracy should focus on ratees rather than rating dimensions. (4) It may be unrealistic to expect raters to accurately make fine discriminations from small samples of behavior. (5) It appears to be very difficult to alter rater ability through training. The positive effects of non-extensive training programs are likely to be on motivation