Exploring mobile market research : the impact of SMS and e-mail invitations on outcome rates of stationary and mobile web surveys

It is the primary focus of the experiment conducted in this study to research the combined impact of survey and invitation mode as experimental factors, on outcome rates of, and hence, participation in web surveys. For this purpose, in a first step, different variables affecting Survey Response Behavior were measured in a profile survey which has been conducted with panelists previously recruited solely for this purpose. A few weeks later, in a second step, all 8,569 panelists were invited by either an invitational SMS or a short e-mail containing a hyperlink to participate in a second survey. The latter has been conducted either on the stationary web, on a PC or laptop, or the mobile web, on a mobile phone. This corresponds to a 2x2 factorial design. Data from the first and second study phase were then combined. Subsequently, an appropriate sample size of 5,124 panelists was drawn for the purpose of evaluation by means of Multiple Binary Logistic Regression Analysis. The results of the profile survey prompt that panelists perceive invitations delivered by e-mail much easier to access than those delivered by SMS, in case they have been previously invited to a stationary web survey. However, in case they were invited to a mobile web survey, they perceive invitations delivered by SMS easier to access than those delivered by e-mail. But which combination of invitation and survey mode will provide researchers with the highest (or lowest) outcome rates? In some combinations of invitation and survey mode panelists are forced to switch between the device they received the invitation on and the one they accessed the survey with, while in other combinations they are not (always) forced to do so. This switch makes web surveys conducted in a particular mode harder to access. Therefore, based on Social Exchange Theory and practical deliberation on panelists’ profile data, it is hypothesized that the anticipated ease of accessing a web survey will have an impact on the outcome rates of web surveys. In this context six hypotheses have been formulated all of which compared one pair of four distinct survey and invitation mode combinations, which served as experimental groups, at a time. Each hypothesis predicted the experimental group which will induce the higher (lower) participation. All hypotheses were tested by means of a Multiple Binary Logistic Regression Analysis. At this, five of six hypotheses regarding the experimental group comparison were not rejected. These results suggested that researchers should always invite panelists to stationary web surveys by e-mail, given that a free choice between above mentioned survey and invitation modes is available to the researcher. Moreover, panelists should be invited by e-mail in case it is expected (for whatever reason) to conduct a stationary web survey. Vice versa it is wise to survey in the stationary web in case invitations by e-mail were taken for granted. In contrast, if the situation requires inviting panelists by SMS it is suggested to conduct a mobile web survey. Likewise, in case a mobile web survey has to be conducted it is advantageousness to invite panelists by SMS. Additional results obtained regarding the covariates included in the analysis indicated that men were to a small extent more likely to participate in this web survey than women. Moreover, the intention to participate in stationary web surveys had a small, and intention to participate in mobile web surveys a very small positive impact on participation. Further, outcome rates also increased with the topic involvement of panelists, but only to a very small extent. Oddly, immediacy of perception had a very small positive impact on participation in regard to e-mails while, in contrast, for SMS it had a very small negative impact. No other covariates showed significant positive or negative impact on outcome rates in this study. Finally, as a secondary focus of research, the four experimental groups have also been compared by means of univariate evaluation regarding other variables of interest. At this, diverse comparisons on data quality, response time, response speed, recommendation behavior, willingness to participate again, as well as all metric variables, constructs and factors measured in the main study have been conducted