Opt-In Online Panel Best Practices

This manuscript provides a brief introduction to opt-in online panels and offers a comprehensive list of procedural principles that can help to improve data quality. The guidelines are organized according to the topics of recruitment, survey instrument quality, identifying careless responses, minimizing dropouts, and preparation for post-survey procedures. Amid the continued expansion of Internet access, strong demand for consumer data in commercial market research, the relatively low costs of Internet-based research, and declining response rates of more traditional phone-based survey research (Pew Research Center, 2012), the use of online surveys/experiments has become popular over the last fifteen years (Baker et al., 2010). However, although online surveys function as a powerful research tool, they are replete with methodological challenges. In particular, many online surveys/experiments that use volunteer, “opt-in panels ” face unique hurdles compared to more traditional probability-based samples and the convenience samples of a university laboratory setting. Opt-in panels consist of individuals who are willing to join a pool of online survey participants, usually in return for compensation like reward points, coupons, being entered in a drawing, or cash. Such participants might be recruited by a banner ad on a website, direct mail, email (Baker et al., 2010), or by word of mouth. Companies that provide opt-in online panels include Qualtrics Panels, Survey Sampling International (SSI), Toluna (formerly Greenfield), and uSamp. There are also a few companies that use variations on opt-in panels. For example, YouGov (formerly Polimetrix) offers what it calls a “matched sample, ” using census data to guide the selection of participants from a large opt-in panel, and constructing weights that the company claims can be used to approximate a representative sample (Rivers, & Bailey, 2009)