†IXA NLP Group, University of the Basque Country

We introduce a distantly supervised event ex-traction approach that extracts complex event templates from microblogs. We show that this near real-time data source is more challeng-ing than news because it contains information that is both approximate (e.g., with values that are close but different from the gold truth) and ambiguous (due to the brevity of the texts), impacting both the evaluation and extraction methods. For the former, we propose a novel, “soft”, F1 metric that incorporates similarity between extracted fillers and the gold truth, giving partial credit to different but similar values. With respect to extraction method-ology, we propose two extensions to the dis-tant supervision paradigm: to address approx-imate information, we allow positive training examples to be generated from information that is similar but not identical to gold values; to address ambiguity, we aggregate contexts across tweets discussing the same event. We evaluate our contributions on the complex do-main of earthquakes, with events with up to 20 arguments. Our results indicate that, de-spite their simplicity, our contributions yield a statistically-significant improvement of 33% (relative) over a strong distantly-supervised system. The dataset containing the knowledge base, relevant tweets and manual annotations is publicly available.