Quality in unimode and mixed-mode designs: a multitrait-multimethod approach

So far, most surveys used face-to-face or telephone questionnaires in order to collect data. But the costs of achieving a survey using these traditional modes increase. At the same time, the response rates decrease, making the idea of switching mode very attractive. Because each mode has its own weaknesses and strengths, the idea of mixing modes of data collection is becoming more and more popular. Nevertheless, combining different modes of data collection may be problematic if people answer differently depending on the mode. Also, a switch from a unimode to a mixed-mode design may threaten the comparability of the data across time. This paper focuses first on the selection effect and shows that different kinds of respondents answer in different modes: therefore, mixing modes might make sense since it may improve the representativeness of the sample keeping the costs low. It is still necessary however to guarantee that mixing modes would not threaten the comparability. Then, the paper therefore compares the quality of questions asked in a unimode and two mixed-mode surveys. Using data of the European Social Survey (ESS) in the Netherlands, and following a multitrait-multimethod approach (MTMM), few differences are found between the unimode and mixed-mode designs in terms of quality. Looking at the differences across modes lead to slightly less similarities, but overall the quality does not change much