What makes a good mix? Chances and challenges of mixed mode data collection in the European Social Survey

Since its inception, the European Social Survey (ESS) has been administered via face-toface interviews. Because of their propensity for yielding high-quality data and high response rates, face-to-face interviews are often considered the gold standard among data collection modes. However, the interrelated challenges of rising survey costs and dwindling response rates have led to doubts about the sustainability of the ESS’s single mode approach. This working paper considers the case for and against the introduction of other modes of data collection, including telephone interviews and web questionnaires, into the ESS. Based on evidence from the scientific literature and from studies conducted under the ESS’s own Mixed Mode Methodology Programme, I argue that (1) there is currently no single-mode alternative for the ESS in any participating country, because incomplete coverage and low response rates rule out administering the survey entirely by telephone or by web questionnaires; but that (2) within-country mixed mode designs, whereby a given country administers the ESS as a mixed-mode survey, may be feasible in some countries. Although they are unlikely to yield higher response rates than single-mode face-to-face designs, within-country mixed mode designs may be able to preserve current response rates while reducing fieldwork costs. For a cross-national time series such as the ESS, a switch from single-mode face-to-face to mixed mode data collection presents a number of specific problems. Variation in surveys modes between countries and over time may lead to mode effects, which would compromise measurement equivalence and diminish the internal validity of crossnational and longitudinal comparisons. A switch to mixed mode data collection should therefore not be undertaken without empirical evidence showing that measurement equivalence can be preserved within a mixed-mode design. I lay out the challenges associated with mixed-mode survey measurement, and argue for a “parallel design” experiment in order to test mixed mode designs in several candidate countries. I also specify methodological conditions that mixed mode designs would have to satisfy in order to have a chance of preserving the high quality of cross-national measurement that the ESS is famous for