On the experience and evidence about mixing modes of data collection in large-scale surveys where the web is used as one of the modes in data collection

This paper presents experience and evidence, from international sources, of the effects on survey quality of the use of mixed modes in data collection, where the web is used as one of the modes. It is recognised that this is an area which is changing fast as the spread of new technologies assist in widening the population who have access and familiarity with using the web. Surveys found to have used mixed modes (including web) were classified around the conventional schema in the literature; according to whether they used concurrent or sequential approaches in their mixed modes and according to whether they were longitudinal or cross-sectional data collections. In addition, surveys are divided according to whether they attempted to survey general populations or sub populations. Existing longitudinal data collection is of particular interest, and especially where data collection modes have changed at some point into the tracking of individuals. Relatively few longitudinal surveys were identified that had attempted to use the web in data collection and no analyses of the consequence of the mode of response on subsequent wave attrition were found. Many of the examples found focused on sub populations of students and young people, who might call the web-savvy populations. This review showed that experience and familiarity with carrying out mixed mode surveys that include the web have grown considerably over the first decade of the 21st century. For experience of surveying general populations, one needs to look to Scandinavian countries, especially the Netherlands. These countries have also been ahead of most others in their prevalence of households with access to the internet. The review of the literature on survey quality effects of mixed modes shows that use of sequential mixed modes, using the web first, followed by more expensive modes, can achieve response rates on a par with good response rates from high quality single mode studies. The different modes recruit samples with different characteristics. However, such approaches, in combining responses from different modes can also gain coverage of the general population that improves on single mode studies. The evidence from experiments also points out how to get the highest web responses at the first stage of a sequential mixed mode data collection series. This is by failing to mention that there are other mode options for the response. However, measurement errors, namely nonresponse errors and mode effects, are likely to be evident in the data collected. These mode effects are still being charted, and are seen to vary according to the type of question, the type of response codes and even the particular topic content; in some cases the mode effects are relatively minor and in other cases substantial. There is also a problem of confounding which is often present with non-response measurement errors or time effects for longitudinal data. There are well documented generalisations about the social desirability consequences and satisficing under different modes. Other research work is continuing to try and identify how to devise questions, by type, that will minimise mode effects. This involves painstaking attention often to the detail of individual questions. Most researchers think the unimode approach to mixed mode question construction is likely to dominate for some time yet, and UK survey fieldwork organisations also tend to adopt this approach