Reliability and accuracy of online panel derived data

There is an increasing trend for researchers to use online panels and crowdsource websites as a source of participants for survey and questionnaire based studies. The driver for this is convenience and capacity to recruit large sample sizes at low cost. There is a stated assumption that online panel respondents are representative of the general population, and that responses are more accurate given their anonymity. Data derived from these samples are used to guide policy decisions, public health initiatives, and treatments. Although sample demographics are often described as representative of the general population, the claim that the responses are more accurate and generalizable is increasingly being drawn into question. A rapid review of the literature since 2012 identified 18 online panel and 7 crowdsource studies, with most published in the previous two years. Results indicated that 60% of these studies failed to report response, or reported participation rather than true response rates. Sixty percent applied a priori or post hoc adjustments to improve data quality. Concerns relate to the high number of participants excluded (30-50%) on the basis of inconsistent responses, rapid completion of questionnaires suggesting inadequate reading of items, and systematic pattern responses. Comparison with reported prevalence rates of gambling disorder ranging from 0.2-1.2% (standardized rate of 2.3%), revealed significantly higher rates of 16.7% to 23.2% for online panels. The limitations of online panels and crowdsource sampling methods will be highlighted and recommendations on procedures to be adopted for data integrity purposes will be described