A comparison of indices for identifying the number of clusters in hierarchical clustering:A study on cognition in schizophrenia patients

Finding clusters in a complex dataset is not straightforward. Different indices were developed to quantify the number of clusters. Their performances were studied using unrealistic simulations, since they were considered at low dimensions. We investigated 14 indices for eight-dimensional data using simulations based on cognition measures. We focused on hierarchical clustering with Ward’s agglomerative technique. Results indicated that Duda and Hart, Hartigan and Gap/pc were best performing. They estimated the number of clusters within ±1 with high probabilities. Duda and Hart index was most consistent, while Gap/pc and WGap/pc together made a good distinction between single and multiple clusters.