Fake It to Make It: A Further Examination of the Susceptibility of Construct-Laden Situational Judgment Tests to Socially Desirable Responding

Faking on personality measures has been a concern for practitioners and academics alike due to the potential resultant reduction in the utility of personality assessments in operational contexts. Researchers have asserted that faking occurs to some extent on most Likert-based noncognitive measures, although the issue of which method of assessment is the most resilient to faking has yet to be settled. A promising option to mitigate concerns regarding faking is the situational judgment test (SJT). SJTs make use of a predetermined scoring key with correct and incorrect answers, making the issue of faking technically moot. Using a 2 (response instruction: faking vs. honest) × 2 (assessment method: SJT vs. single-statement) experimental design with response format nested within the SJT (rate vs. rank) and single-statement (Likert vs. true-false) methods, the comparative susceptibility of SJTs and single-statement measures to faking was examined. It was hypothesized that the SJT would be more resilient to attempts to “fake good” than single-statement measures of the same constructs when test-takers are given explicit instructions to fake. It was also hypothesized that the rank SJT response format would be more resilient to faking compared to the rate SJT format. In a sample of 583 participants recruited from Amazon Mechanical Turk (MTurk), the results indicated that across response formats, the standardized mean difference between the honest and faking conditions on tests of agreeableness and conscientiousness were significantly larger for the single-statement measures compared to the SJT. Both the rate and rank SJTs were more resilient to faking than the Likert and true-false single-statement measures. In addition, the rank SJT was more resilient to faking than the rate SJT; however, this effect was obtained for the conscientiousness SJT but not agreeableness. Using supervisor-perspective job performance ratings, no support was found for any of the hypotheses positing that the SJT measures would have higher criterion-related validity than the single-statement measures under faking conditions. Overall, the results indicated that the SJTs were effective at preventing mean shifts due to faking, but the higher resiliency to faking did not result in higher criterion-related validity compared to the single-statement measures. The implications for the science and practice of SJTs, and faking on noncognitive measures, specifically personality, are discussed