2013a). Crowd Labor Markets as Platform for IS Research: First Evidence from Electronic Markets

Crowd labor markets such as Amazon Mechanical Turk (MTurk) have emerged as popular platforms where researchers can inexpensively run web-based experiments. Recent work even suggests that MTurk can be used to run large-scale field experiments such as prediction markets in which participants interact synchronously in real-time. Besides technical issues, several methodological questions arise and lead to the question of how results from MTurk and laboratory experiments compare. In this work we provide first insights into running market experiments on MTurk and compare the key property of markets, information efficiency, to a laboratory setting. The results are mixed at best. On MTurk, information aggregation took place less frequently than in the lab. Our results suggest that MTurk participants cannot handle as much complexity as laboratory participants in time-pressured, synchronized experiments