Examining the fakability of forced-choice individual differences measures

Forced-choice (FC) is a popular format for developing noncognitive individual differences measures, where individuals are forced to choose one or multiple statements out of several options. FC measures have been proposed as a valuable approach to reduce score inflation in high-stakes assessments, but their effectiveness has not yet been examined in a meta-analysis. In my dissertation, I conducted two studies to examine the fakability of forced-choice personality and vocational interest measures. In the first study, I conducted a meta-analysis of studies comparing FC personality measures between low-stakes and high-stakes situations. Results suggested that the overall score inflation effect size for FC personality measures is 0.05, which is much lower than the effect size for single-statement measures. The score inflation effect size was also found to vary across FC scale characteristics, study design factors, and personality facets. Specifically, a faking resistant FC scale should be constructed in a multidimensional PICK format, with statements balanced in social desirability and responses scored in a normative approach. Personality facets of high relevance to the target were found to exhibit larger inflation than facets of low relevance to the target job. In the second study, I conducted an induced faking study to examine the fakability of vocational interest measures and whether or not the FC format reduced the faking effect. With a sample of 1,559 respondents, I found that respondents consistently inflated the interest domain that matched the target job, and that the inflation led to a decreased level of criterion-related validity of interest-job fit. Multidimensional FC scales exhibited a smaller inflation effect on non-matched domains, but not on matched domains, than single-statement scales. However, the FC format did not reduce the level of criterion-related validity attenuation found in single-statement measures