Reading Between the Tweets: Clustering and Topic Extraction on Micro-Blogs

Additional contributor: Daniel Boley (faculty mentor).According to Java et al., some of the main intentions of users on Twitter are daily chatting, conversations, sharing information/URLs, and reporting news. Topic extraction has been used in large documents such as online news articles, emails, scientific literature, and blogs, but little has been done with short documents like micro-blogs. Objective: Apply clustering and topic extraction techniques to micro-blogs as a first step toward tracking popular topics dynamically as they vary by time or location, or to enhance topic-related searches of tweets. The results found using a hybrid PDDP and K-Means clustering technique, typically used for categorizing large documents, have shown success when working with micro-blogs with noisy dictionaries. The methods show potential to be a basis for [semi-]supervised topic tracking and extraction using methods previously used for larger text documents.This research was funded by the Undergraduate Research Opportunities Program (UROP)