Mining News Content for Popularity Prediction

The problem of popularity prediction has been studied extensively in various previous research. The idea behind popularity prediction is that the attention users give to online items is unequally distributed, as only a small fraction of all the available content receives serious users attention. Researchers have been experimenting with different methods to find a way to predict that fraction. However, to the best of our knowledge, none of the previous work used the content for popularity prediction; instead, the research looked at other features such as early user reactions (number of views/shares/comments) of the first hours/days to predict the future popularity. These models are built to be easily generalized to all data types from videos (e.g. YouTube videos) and images, to news stories. However, they are not considered very efficient for the news domain as our research shows that most stories get 90% to 100% of the attention that they will ever get on the first day. Thus, it would be much more efficient to estimate the popularity even before an item is seen by the users. In this thesis, we plan to approach the problem in a way that accomplishes that goal. We will narrow our focus to the news domain, and concentrate on the content of news stories. We would like to investigate the ability to predict the popularity of news articles by finding the topics that interest the users and the estimated audience of each topic. Then, given a new news story, we would infer the topics from the story’s content, and based on those topics we would make a prediction for how popular it may become in the future even before it’s released to the public