Dimensionality Reduction with Generalized Linear Models

In this paper, we propose a general dimensionality reduction method for data generated from a very broad family of distributions and nonlinear func-tions based on the generalized linear model, called Generalized Linear Principal Component Analysis (GLPCA). Data of different domains often have very different structures. These data can be mod-eled by different distributions and reconstruction functions. For example, real valued data can be modeled by the Gaussian distribution with a lin-ear reconstruction function, whereas binary valued data may be more appropriately modeled by the Bernoulli distribution with a logit or probit func-tion. Based on general linear models, we propose a unified framework for extracting features from data of different domains. A general optimization algo-rithm based on natural gradient ascent on distribu-tion manifold is proposed for obtaining the maxi-mum likelihood solutions. We also present some specific algorithms derived from this framework to deal with specific data modeling problems such as document modeling. Experimental results of these algorithms on several data sets are shown for the validation of GLPCA.