Detecting Newsworthy Topics in Twitter

The task of the SNOW 2014 Data Challenge is to mine Twitter streams to provide jour-nalists a set of headlines and complementary information that summarize the most news-worthy topics for a number of given time in-tervals. We propose a 4-step approach to solve this. First, a classifier is trained to determine whether a Twitter user is likely to post tweets about newsworthy stories. Second, tweets posted by these users during the time inter-val of interest are clustered into topics. For this clustering, the cosine similarity between a boosted tf-idf representation of the tweets is used. Third, we use a classifier to estimate the confidence that the obtained topics are news-worthy. Finally, for each obtained newswor-thy topic, a descriptive headline is generated together with relevant keywords, tweets and pictures. Experimental results show the effec-tiveness of the proposed methodology. Copyright c © by the paper’s authors. Copying permitted only for private and academic purposes