Measurement Invariance of Assessment Center Ratings: Consistency of Dimensional Constructs across Exercises

The criterion-related validity of assessment centers (ACs) has been consistently supported. However, there has been an ongoing debate about AC construct validity in regard to low dimension variance and overriding exercise variance. Many studies that showed weak dimension effects in ACs have been based on the multitrait-multimethod (MTMM) framework which posits cross-situational variance as error and all capability dimensions can be equally represented from different exercises. The goal of this study is to introduce a measurement invariance (MI) framework for evaluating the construct validity of AC dimensions. Specifically, MI analysis was used to test whether the construct meaning of AC dimensions remains consistent across different AC exercises. The analyzed ACs in which six dimensions were assessed each in four exercises showed invariance of factor loadings across two to four exercises with or without specification of a single overall ability factor with few exceptions. Dimension ratings from particular exercises were systematically higher or lower than those from other exercises while certain exercises were more relevant to specific dimensions. The findings were in support of apparent dimension factor in the ACs and need for careful interpretation of exercises effect