A system of topic mining and dynamic tracking for social texts

A massive amount of information is stored as text in the real world. Classifying the texts according to topics is an approach for people to extract useful information. Social medias generate a mass of texts every day. Topic mining and tracking on social texts are beneficial to both humanity and IT areas. Although ready-made algorithms for topic mining and evolution tracking exist, existing methods are mostly aimed at static data and only to the mining phase of the topics. There is a lack of a general and entire solution covering all phases of topic mining and tracking of social texts. This thesis aims to develop an entire and coherent system which can receive social texts from real-time data streams, mine topics from texts and track topic evolution over time. It is based on the existing algorithms. Tests were conducted after the development, including coverage of LDA for social texts, performance of system and presentation of system in the real environment. According to the experiment results, the system operated smoothly in the real environment. The existing algorithms are effective to social texts. The system successfully covered the whole process of topic mining for social texts as expected. However, there is still room for system improvement. Since the system is a prototype, there may be a need to change it based on requirements of the real application if the system is put into practice and a lot of real tests should be performed in order to guarantee it is functioning well