Estimation of Measurement Error and Identification of Causes: Linking Measurement Error to Nonresponse, Interviewers, and Interviewer Characteristics

In order to identify survey design factors that induce measurement error, validation data are needed. Gold standards are difficult to obtain and are of limited types. In the absence of truth, analyses commonly revert to differences in means, such as the Kish and Hansen, Hurwitz, and Bershad models for interviewer variance, and similar ANOVA models for comparisons of response scales. Proposed is a two-stage iterative approach to estimating and modeling causes of measurement error the measurement error is first estimated through a predictive model and then it is modeled using hypothesized causes. This two-stage model is demonstrated with data from the 2002 National Election Study to examine interviewer-induced measurement error, and its relationship to proxies for unit nonresponse. Effects of interviewer experience and characteristics on measurement error are estimated. Results show that measurement error variance induced by the interviewers is quite unrelated to interviewer variance that is defined as differences in their means. Furthermore, estimates of differences in their means and their standard errors are biased when interviewer measurement error variances are assumed to be the same across interviewers. Findings are consistent with prior studies of interviewer effects, as the measurement error variance was more than four times larger for black respondents interviewed by white interviewers than when interviewed by black interviewers, on thermometer ratings for blacks. The estimated measurement error was also found to be positively associated with indicators of unit nonresponse.