Interviewer Falsification in Survey Research: Detection Methods and Impact of Fraudulent Interviews

The role of the interviewer and sources of interviewer error in the survey data collection process are widely discussed topics in the survey methodology literature. An understudied problem in this context is the falsification of interview data by the interviewer. Research gaps concern, for example, how conclusions drawn from survey data are affected by falsified interviews. So far it is commonly assumed, that the possible effect of falsifications on univariate statistics can only be as high as the overall share of falsified data. Since the share of faked data is usually very low for most surveys, the problem is regarded as negligible. However, this advice is unlikely to hold for all types of survey questions. For bi- and multivariate statistics, the impact of interviewer falsification has also received little research attention. A further research gap concerns the utility and sensitivity of different identification methods proposed in the literature. Overall, the number of proposed identification methods is quite considerable, and numerous falsification indicators are suggested by different authors. Such indicators attempt to distinguish between honest and fake interviewing behaviours as well as the responding styles of real and fictional respondents. These indicators can be differentiated between formal indicators (analyzing the response behavior), content related indicators (analyzing the distribution of different items) as well as indicators on paradata (analyzing differences within the paradata). These indicators can be incorporated into a variety of statistical methods (e.g. cluster analysis) that attempt to identify “at risk” interviewers. However, it is rare to find empirical evaluations of these methods in real-world settings. The above research gaps are partially due to the lack of appropriate empirical data made publicly available to researchers. A large proportion of studies are based solely on experimental (or simulated) data, for which the universal validity of the results are questionable. Using actual fake interview data from a large survey in Germany, we investigated and sought to address the above research gaps. These data include a variety of different question types that permit a broad and systematic comparison of the described methods. Hence, this paper aims to 1) identify the potential biases introduced by interviewer falsification, and 2) evaluate the potential usefulness of various identification strategies proposed in the literature. In addition, we propose new falsification detection methods to improve future interviewer controls