Topic Detection, Tracking and Trend Analysis Using Self-Organizing Neural Networks

We address the problem of Topic Detection and Tracking (TDT) and subsequently detecting trends from a stream of text documents. Formulating TDT as a clustering problem in a class of self-organizing neural networks, called the Adaptive Resonance Theory (ART) networks, we propose an incremental algorithm to solve this clustering problem. From the topics being detected and tracked, we showhow trends can be identified. From our experimental studies, we find that our algorithm has been able to detect hot topics automatically and trackthem to a good accuracy. The method is also observed to enable discovering interesting trends that are not directly mentioned in the individual documents but deducible only from reading all relevant documents