Asking for numbers and quantities: Visual design effects in web surveys. Paper presented at the Paper presented at the AAPOR Annual Conference

When reporting frequencies and quantities in self-administered surveys using open-ended questions, respondents make use of estimation strategies in order to provide a reasonable response. Accordingly, respondents do not only report pure numbers using digits, instead they extend their response to alphanumeric information and explicit estimates (e.g. ‘‘about 10–20 cigarettes per day’’). Such responses are not only of reduced quality, but they also require a lot of editing and cleaning in the post-field phase of a survey. While previous methodological work on frequency questions has focused on question wording, reference periods and other text elements that have traditionally been considered significant for high quality frequency responses, in this article we test the impact of various graphical characteristics of the response field in order to reduce the number of alphanumeric elaborations and explicit estimates. The field-experimental studies reported are based on the assumption that respondents interpret formal characteristics of frequency questions as meaningful information throughout the question answer process. The findings reported will help survey designers reduce the proportion of estimated responses. In almost every self-administered survey, respondents are asked to report frequencies or quantities of behaviors or events in a given time frame or reference period. Some classic examples are: ‘‘During the past 12 months, how many times have you seen a doctor or other health care professional about your own health at a doctor’s office, a clinic, or some other place?’ ’ or ‘‘On average how many cigarettes do you smoke a day1?’’. Respondents answer this type of question by way of two response strategies: They either ‘‘recall and count’ ’ the instances of the desired behavior or event, or they ‘‘estimate’ ’ th