Expectation-Maximization for Sparse and Non-Negative PCA

We study the problem of finding the dom-inant eigenvector of the sample covariance matrix, under additional constraints on the vector: a cardinality constraint limits the number of non-zero elements, and non-negativity forces the elements to have equal sign. This problem is known as sparse and non-negative principal component analysis (PCA), and has many applications includ-ing dimensionality reduction and feature se-lection. Based on expectation-maximization for probabilistic PCA, we present an al-gorithm for any combination of these con-straints. Its complexity is at most quadratic in the number of dimensions of the data. We demonstrate significant improvements in per-formance and computational efficiency com-pared to other constrained PCA algorithms, on large data sets from biology and com-puter vision. Finally, we show the usefulness of non-negative sparse PCA for unsupervised feature selection in a gene clustering task. 1