New Evidence On the Construct Validity of Diagnostic Assessment Centers: The Findings May Not Be So Troubling After All

Research continues to demonstrate that assessment centers designed for administrative purposes lack construct validity. Unfortunately, little research has investigated whether this trend generalizes to developmental applications. This study sought to extend past research by using multitrait-multimethod analysis and confirmatory factor analysis (CFA) to examine the construct validity of a center designed for diagnostic purposes. CFA was used to evaluate the plausibility of four competing models of assessment center ratings which varied according to their degree of cross-situational consistency and situational specificity. Overall, the findings were mixed. While the multitrait-multimethod data were generally consistent with past research, the CFA evidence was not. Specifically, results revealed that assessors\u27 ratings of 138 first-year MBA students represented both dimension and exercise factors. This suggests that assessors were able to distinguish among the various dimensions measured by the situational exercises. Stated differently, assessors appeared to organize information in terms of both dimensions and exercises. Implications of the findings and suggestions for future research are addressed