Clustering of High-Dimensional and Correlated Data

Abstract Finite mixture models are being commonly used in a wide range of ap-plications in practice concerning density estimation and clustering. An attractive feature of this approach to clustering is that it provides a sound statistical frame-work in which to assess the important question of how many clusters there are in the data and their validity. We consider the applications of normal mixture models to high-dimensional data of a continuous nature. One way to handle the fitting of normal mixture models is to adopt mixtures of factor analyzers. However, for ex-tremely high-dimensional data, some variable-reduction method needs to be used in conjunction with the latter model such as with the procedure called EMMIX-GENE. It was developed for the clustering of microarray data in bioinformatics, but is applicable to other types of data. We shall also consider the mixture procedure EMMIX-WIRE (based on mixtures of normal components with random effects), which is suitable for clustering high-dimensional data that may be structured (cor-related and and replicated) as in longitudinal studies.