Incremental knowledge discovery in social media

In light of the prosperity of online social media, Web users are shifting from data consumers to data producers. To catch the pulse of this rapidly changing world, it is critical to transform online social media data to information and to knowledge. This dissertation centers on the issue of modeling the dynamics of user communities, trending stories, topics and user interests in online social media. However, knowledge discovery and management in online social media is challenging because: 1) social media data arrive in the form of continuous streams; 2) the volume of social media data is potentially infinite; and 3) more importantly, social media data is very complex which consists of network, text, tag, click and many other information.To achieve the abovementioned research goal, this dissertation proposes a research framework with three steps. First of all, this dissertation aims at detecting evolving user communities because of its broad applications in e-commerce, online social media, intelligent security, public health and more. We propose a Dynamic Stochastic Blockmodel with Temporal Dirichlet Process, which enables the detection of user communities and tracks their evolution simultaneously from a network stream. The number of user communities is automatically determined by a Recurrent Chinese Restaurant Process without any human intervention. In addition, the identified communities exhibit a rich-gets-richer effect and other beneficial properties. The experiment evaluations suggested the effectiveness of the proposed model.Secondly, instead of considering only network data, we take textual information and user interest into account to design a novel probabilistic graphical model which can detect interpretable and dynamic trends and topics from document streams, where each trend (short for trending story) corresponds to a series of continuing events or a storyline. Experiments on three different datasets indicated that our proposed model can capture meaningful topics and trends, monitor rise and fall of detected trends, and outperform baseline approach in terms of the perplexity on held-out dataset.Last but not least, to study the benefit of trend and topic modeling, we propose to adopt the user interests learnt from social media documents, which are typically contributed collaboratively by Web users, to improve the performance of recommendation in social media. To achieve this research goal, we introduced two reranking models, which combine the power of bipartite graph based collaborative filtering and user interest detected by a probabilistic graphical model, to rank social media documents. Our proposed approaches successfully make use of the user interest extracted from social media data to address the cold star issue. Experiment results on a public dataset demonstrated the effectiveness of the proposed techniques.Ph.D., Information Studies -- Drexel University, 201