Exploring the impacts of negative emotional reactions to the visual design of web surveys

Due to recent scholarly attention to visual design, much is known about the effects that specific design features have on web surveys, but little attention has been paid to the effects of overall screen design. Applying Norman's (2004) emotional design concepts to a web survey experiment, we proposed and tested hypotheses related to potential data quality impacts expected when screen designs elicit negative emotional reactions. Such reactions were likely to stem from designs that were displeasing at the visceral or behavioral levels, so this experiment compared data collected from a sample of 1800 undergraduates completing three versions of the same Student Experience Survey. Results produced by the viscerally- and behaviorally-displeasing experimental designs were each compared with those from a control form, representing good emotional design. Altered moods, perceptions, motivation, and cognitive functioning were expected to translate into reduced data quality along four avenues: respondent cooperation, substantive response patterns, respondent commitment, and subjective survey experience. Both experimental forms produced lower quality data the majority of indicators examined, with at least some evidence of an effect on each aspect of data quality. Still, some measures showed mixed results, and a number of predictions were unconfirmed. Therefore, a series of in-depth cognitive interviews of approximately 30 Washington State University undergraduates were conducted in order to elucidate those nuanced differences and provide a deeper understanding of how screen designs affect emotional reactions and how those reactions translate to response patterns. These interviews provided a three-dimensional picture of the effects, as well as the processes by which visual qualities of web surveys impact respondents' behavior. This methodological triangulation theoretically-validated and improved analysis of the field tests, while allowing us to ascertain unexpected effects and refine our theoretical expectations for further research. Based on both qualitative and quantitative findings, it is now clear that screen design alone can impact respondents' emotional reactions, altering behavior and cognition, and potentially reducing data quality in a number of ways. However, it is also now clear that other visual processing effects, as well as non-visual characteristics of the survey and the sample frame may mitigate these effects