Interactive Classification of Very Large Datasets with BIRCH

Finding useful patterns in large datasets has attracted considerable interest recently, and one of the most widely studied problems in this area is the identification of clusters in a multi-dimensional dataset. Prior work has mostly been in the Statistics and Machine Learning communities, and does not adequately address the problem of large datasets and minimization of I/O costs. BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies), is a data clustering algorithm especially suitable for very large datasets. BIRCH incrementally and dynamically clusters incoming multi-dimensional metric data points. BIRCH can typically find a good clustering with a single scan of the data, and improve the clustering quality further with a few additional scans. It adjusts dynamically to the input dataset to try to produce the best quality clustering with the available resources (i.e., available memory and time constraints). In this paper, we demonstrate that BIRCH can be integrated with a..