Response rate and response error in marketing research

In this study, I investigate the perspectives of marketing researchers views about the two important concepts of survey research, response rate and response bias. I specifically aimed to answer the following questions: Research Question #1: Do marketing researchers separate the concepts of response rate from response bias? Research Question #2: How exactly should data quality be measured? Is it about sample representativeness, minimizing non- response bias or just solely Increasing the response rate? Research Question #3: What are researchers doing to assess and minimize response bias? Research Question #4: Do additional efforts put forth by survey researchers, such as reminder letters and incentives, for the purpose of increasing survey response rate affect additional sampling bias? Research Question #5: Are the common techniques used by survey researchers to increase response rate equally effective? In an attempt to answer the research questions stated, I have collected both primary and secondary data. The primary data was collected from the Academy of Marketing Science active members. Eight versions of an excerpt was prepared that is taken from an actual article that was accepted for publication recently. The experimental design adopted was 2x2x2 with each treatment trying to answer one of the research questions stated in this study. The first treatment was regarding the consideration of population. The subjects were manipulated with two versions of the excerpt of which one was with a Canadian sample and the second was with a so called North American sample. The second treatment was about the manipulation of the initial number of surveys sent out which as a result would change the response rate percentage. The two different versions included 500 vs. 5000 initial surveys sent out varying the response rate from 5.1% to 50.2%. The third treatment included the manipulation of Armstrong and Overton (1977) citation. The over use of this citation in the marketing literature has been noteworthy. In an attempt to investigate this matter further, this treatment was created. First version contained a sentence that stated that the early and the late respondents were compared and no significant differences were found as evidence of no response bias including the citation of Armstrong and Overton (1977). The second version of the excerpt included a table with the expected demographics regarding the population of interest. In addition to these, the subjects were also assigned to two different conditions where they were asked to evaluate the excerpt as an author or as a reviewer. In the invitation email respondents were asked to toss a coin or to click on a web link that would toss the coin for them and select the appropriate link that corresponds with their choice. In order to assess the popular techniques of enhancing response rate, I have divided the sample into several groups as the pre-notification, the reminder and the pre- notification and reminder groups and a control group that received neither the reminder nor the pre-notification treatment. The results revealed that, according to my sample none of the techniques mentioned above, improve the response rate. The secondary data was collected from major outlets of the marketing science (Journal of Marketing-JM, Journal of Marketing Research-JMR, and Journal of the Academy of Marketing Science -JAMS) during the periods of 2005-2010. The final sample consisted of 68 JM, 23 JMR and 84 JAMS articles. In addition to these, I also randomly selected 31 rejected articles from the Journal of Business Research (JBR) archives. The results of the study revealed that, survey researchers do not clearly grasp the concepts of response rate and response bias. In addition, the results demonstrated that the data quality should be measured by the sample\u27s representativeness of the population and the researcher\u27s capability of decreasing the response and the non-response biases. Further, the techniques used to enhance response rate such as reminder and prenotification letters as well as incentives are not as effective and are likely to introduce additional response bias to a study. The results also showed that the optimal data collection method researchers should consider adopting is the combination method. The study ends with the discussion of the implications of these results and possible future extensions of this study