Adaptive sampling and fast low-rank matrix approximation

Abstract

We prove that any real matrix A contains a subset of at most 4k/ε + 2klog(k + 1) rows whose span "contains" a matrix of rank at most k with error only (1 + ε) times the error of the best rank-k approximation of A. We complement it with an almost matching lower bound by constructing matrices where the span of any k/2ε rows does not "contain" a relative (1 + ε)-approximation of rank k. Our existence result leads to an algorithm that finds such rank-k approximation in time O(M(k/ε + k 2log k)+(m + n)(k2/ε2 + k3 log k/ε + k4 log2k)), i.e., essentially O(Mk/e), where M is the number of nonzero entries of A. The algorithm maintains sparsity, and in the streaming model [12, 14, 15], it can be implemented using only 2(k + 1)(log(k + 1) + 1) passes over the input matrix and O (min{m, n}(k/ε + k2 log k)) additional space. Previous algorithms for low-rank approximation use only one or two passes but obtain an additive approximation. © Springer-Verlag Berlin Heidelberg 2006.