The effects of response alternative precision and cognitive reflection on biased responding in questionnaires

Krosnick and Fabrigar (1997) believe that using precise verbal labels of response alternatives in questionnaires will decrease the difficulty of the questionnaire and therefore decrease biased responses. Empirical evidence has not yet been found for this assumption, therefore the objective of this study is to observe whether three types of response biases; acquiescent and disacquiescent responses, as well as the sum of the previous two (named misresponding) are more prevalent among respondents who receive imprecise verbal labels of response alternatives in a survey than among those who receive precise verbal labels. Furthermore the effects of cognitive reflection on that relationship will be examined. Participants were randomly assigned into two groups, 220 participants received a survey that had precise verbal labels of response alternatives, and 265 participants received a survey with imprecise verbal labels. Logistic regression revealed that participants in the imprecise group were more likely to respond with disacquiescence than participants in the precise group. The same effect was not found for acquiescence or misresponding. An interaction effect was found between cognitive reflection and precision in disacquiescence as well as in misresponding. These results indicate that Krosnick's and Fabrigar's hunch about the importance of verbal label precision in making the response task easier was at least partly right. This also indicates that cognitive reflection plays a role in deciding what determines the difficulty of the response task