Lower bounds for sparse coding

Abstract

We give lower bounds on the reconstruction error for PCA, k-means clustering, and va_absrious sparse coding methods. It is shown that the two objectives of good data approximation and sparsity of the solution are incompatible if the data distribution is evasive in the sense that most of its mass lies away from any low dimensional subspace. We give closure properties and examples of evasive distributions and quantify the extent to which distributions of bounded support and bounded density are evasive.