Electronic Colloquium on Computational Complexity, Report No. 42 (2006) Adaptive Sampling and Fast Low-Rank Matrix Approximation

We prove that any real matrix A contains a subset of at most 4k/ɛ+2k log(k+1) rows whose span “contains ” a matrix of rank at most k with error only (1+ɛ) times the error of the best rankk approximation of A. This leads to an algorithm to find such an approximation with complexity essentially O(Mk/ɛ), where M is the number of nonzero entries of A. The algorithm maintains sparsity, and in the streaming model, it can be implemented using only 2(k + 1)(log(k + 1) + 1) passes over the input matrix. Previous algorithms for low-rank approximation use only one or two passes but obtain an additive approximation.