An Evaulation of the Criterion Measures in a Leadership Assessment Center

The development process for any performance appraisal system, including assessment centers, is critical to the validity and utility of the system. The current research consisted of two studies evaluating the criterion measures of a leadership assessment center at a southeastern university. The first study examined the rating scale development process and compared the shortcut method to the traditional method for developing Behaviorally Anchored Rating Scales (BARS). The second study examined interrater agreement for criterion ratings in a leadership assessment center. Results from Study 1 demonstrated that the shortcut method for developing BARS resulted in scales with different properties from those developed with the traditional method. The results of Study 2 suggest raters have higher agreement within an exercise than within competencies across exercises. Researchers should further investigate the implications of abandoning competencies and examine if placing more of an emphasis on Performance Dimension Training can improve the ratings of competencies