Mining hierarchies of correlation clusters

The detection of correlations between different fea-tures in high dimensional data sets is a very important data mining task. These correlations can be arbitrar-ily complex: One or more features might be correlated with several other features, and both noise features as well as the actual dependencies may be different for dif-ferent clusters. Therefore, each cluster contains points that are located on a common hyperplane of arbitrary dimensionality in the data space and thus generates a separate, arbitrarily oriented subspace of the original data space. The few recently proposed algorithms de-signed to uncover these correlation clusters have sev-eral disadvantages. In particular, these methods can-not detect correlation clusters of different dimension-ality which are nested into each other. The complete hierarchical structure of correlation clusters of vary-ing dimensionality can only be detected by a hierar-chical clustering approach. Therefore, we propose the algorithm HiCO (Hierarchical Correlation Ordering), the first hierarchical approach to correlation clustering. The algorithm determines the cluster hierarchy, and visualizes it using correlation diagrams. Several com-parative experiments using synthetic and real data sets show the performance and the effectivity of HiCO. 1