Why Is It Difficult to Detect Outbreaks in Twitter?

In this paper, we present an event-based Epidemic Intelli-gence (EI) system framework leveraging social media data, e.g., Twitter messages (or tweets) for providing public health officials the necessary tools to survey and sift through rele-vant information, namely, disease outbreak events. There exists three main research challenges in gathering epidemic intelligence from social media streams: 1) dynamic classifi-cation to enable message filtering, 2) signal generation pro-ducing reliable warnings based on observed term frequency changes in the filtered messages, and 3) providing search and recommendation functionalities to domain experts, for better assessment of the potential outbreak threats associ-ated with the generated signals. We outline possible ap-proaches to solve these important challenges as well as dis-cuss areas where further research is required. The aim of this paper is to provide guidance for similar endeavors, and to give prospective event-based Epidemic Intelligence system builders a more realistic view on the benefits and issues of social media stream analysis. 1