CROWD-SOURCED CODING OF POLITICAL TEXTS*

Empirical social science often relies on data that are not observed in the field, but are coded into quantitative variables by expert researchers who base their codes on qualitative raw sources. Using crowd-sourcing to distribute text coding to massive numbers of non-experts, we generate results comparable to those from expert coding, but far more quickly and flexibly. Crucially, the data we collect can be reproduced or extended cheaply and transparently, making crowd-sourced datasets intrinsically replicable. This focuses researchers ’ attention on the fundamental scientific objective of specifying reliable and replicable methods for collecting the data needed, rather than on the content of any particular dataset. The findings reported here concern text coding, but have general implications for expert coded data in the social sciences. 11,687 word count after the front page * An earlier draft of this paper, with much less complete data, was presented at the third annual Analyzing Text a