Error-Correction and Aggregation in Crowd-Sourcing of Geopolitical Incident Information

Abstract. A discriminative model is presented for crowd-sourcing the annotation of news stories to produce a structured dataset about inci-dents involving militarized disputes between nation-states. We used a question tree to gather partially redundant data from each crowd worker. A lattice of Bayesian Networks was then applied to error correct the indi-vidual worker annotations, the results of which were then aggregated via majority voting. The resulting hybrid model outperformed comparable, state-of-the-art aggregation models in both accuracy and computational scalability.