Simultaneous Clustering and Dimensionality Reduction Using Variational Bayesian Mixture Model

Summary. Exponential principal component analysis (e-PCA) provides a frame-work for appropriately dealing with various data types such as binary and integer for which the Gaussian assumption on the data distribution is inappropriate. In this paper, we develop a simultaneous dimensionality reduction and clustering technique based on a latent variable model for the e-PCA. Assuming the discrete distribution on the latent variable leads to mixture models with constraint on their parameters. We derive a learning algorithm for those mixture models based on the variational Bayes method. Although intractable integration is required to implement the algo-rithm, an approximation technique using Laplace’s method allows us to carry out clustering on an arbitrary subspace. Numerical experiments on synthetic data and the MNIST handwritten digits data demonstrate its effectiveness for extracting the structures of data as a visualization technique and its high generalization ability a