Gaussian mixture model with local consistency

Gaussian Mixture Model (GMM) is one of the most popular data clustering methods which can be viewed as a linear com-bination of different Gaussian components. In GMM, each cluster obeys Gaussian distribution and the task of clustering is to group observations into different components through estimating each cluster’s own parameters. The Expectation-Maximization algorithm is always involved in such estima-tion problem. However, many previous studies have shown naturally occurring data may reside on or close to an un-derlying submanifold. In this paper, we consider the case where the probability distribution is supported on a sub-manifold of the ambient space. We take into account the smoothness of the conditional probability distribution along the geodesics of data manifold. That is, if two observations are “close ” in intrinsic geometry, their distributions over dif-ferent Gaussian components are similar. Simply speaking, we introduce a novel method based on manifold structure for data clustering, called Locally Consistent Gaussian Mixture Model (LCGMM). Specifically, we construct a nearest neigh-bor graph and adopt Kullback-Leibler Divergence as the “dis-tance ” measurement to regularize the objective function of GMM. Experiments on several data sets demonstrate the ef-fectiveness of such regularization