On-Line New Event Detection using Single Pass Clustering

This paper discusses the implementation and evaluation of a new-event detection system. We focus on a strict on-line setting, in that the system must indicate whether the current document contains or does not contain discussion of a new event before looking at the next document. Our approach to the problem uses a single pass clustering algorithm and a novel thresholding model that incorporates the properties of events as a major component. A corpus containing newswire and transcribed broadcast news was analyzed using our system, and our results compared favorably to those of other systems. We develop an evaluation methodology based on a combination of techniques that allows us to infer the expected performance of our approach in the field, and to suggest avenues for future research that may lead to better performance. 1 Introduction On-line event detection is the classification of a timeordered stream of documents that discuss events. The task of new-event detection is to identify doc..