Construct validity of multi-source performance ratings: An examination of the relationship of self-, supervisor-, and peer-ratings with cognitive and personality measures

Although more and more organizations prefer using multi-source performance ratings or 3601 feedback over traditional performance appraisals, researchers have been rather skeptical regarding the reliability and validity of such ratings. The present study examined the validity of self-, supervisor-, and peer-ratings of 195 employees in a Dutch public organization, using scores on an In-Basket exercise, an intelligence test, and a personality questionnaire as external criterionmeasures. Interrater agreement ranged from.28 to.38. Variance in the ratings was explained by both method and content factors. Support for the external construct validity was rather weak. Supervisor-ratings were not found to be superior to self- and peer-ratings in predicting the scores on the external measures. P erformance feedback in an organizational setting bymultiple sources (e.g., supervisor, peers, subordinates, and self), or 3601 feedback, is enjoying great popularity. An increasing number of organizations have started using some kind of multi-source performance feedback (Church & Bracken, 1997; London & Smither, 1995). Estimates of the percentage of organizations in the United States usin