Using mode preference to disentangle nonresponse and measurement effect in a sequential mode survey

The aim of this research is to assess if variables on mode preference can be used in propensity score model to control for selection effects in a sequential mixed mode in order to evaluate measurement effects. Survey methodologists hope that mixed mode can be implemented as a solution to the problem of increasing survey costs, decreasing response rates and risk of nonresponse bias. The disadvantage of such survey designs is the lack of control of mode-related measurement effects that can put in doubt the comparability with previous rounds or waves conducted in a single (other) mode survey. To understand the extend of measurement effects, selection and measurement effects between modes have to be disentangled. Almost all existing techniques to disentangle both effects are dependent on a set of covariates that are assumed to be mode invariant and explain in a sufficient way selection effects. Finding such a set of covariates has proven to be a challenge. In this paper, we investigate a set of variables that has been included in the ESS sequential mixed mode experiment in Estonia, a web component (mode b) with a face-to-face follow-up (mode a). A supplementary questionnaire contained three items about on mode preference (preference of participation in web/phone/face-to-face survey). We will use the mode preference items on top of the available socio-demographical variables to control for selection effects. These mode preference variables are expected to explain to a larger extend the selection effect than socio-demographical variables but may be subject to measurement effects. This means that these variables are more mode-sensitive than socio-demographical variables. The trade-off between the compliance to the two assumptions, mode insensitivity and mode-selection ignorability, and the resulting estimated selection and measurement effects are considered. We calculate propensity score weighting based on only socio-demographic variables and adding the mode preference items to equate the web and face-to –face respondent samples and evaluate measurement effects.status: publishe