Tight bounds for `p oblivious subspace embeddings

Abstract

An `p oblivious subspace embedding is a distribution over r × n matrices Π such that for any fixed n × d matrix A, Pr Π [for all x, kAxkp ≤ kΠAxkp ≤ κkAxkp] ≥ 9/10, where r is the dimension of the embedding, κ is the distortion of the embedding, and for an n-dimensional vector y, kykp = Pni=1 |yi|1/p is the `p-norm. Another important property is the sparsity of Π, that is, the maximum number of nonzero entries per column, as this determines the running time of computing Π·A. While for p = 2 there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparsity, for the important case of 1 ≤ p < 2, much less was known. In this paper we obtain nearly optimal tradeoffs for `p oblivious subspace embeddings for every 1 ≤ p < 2. Our main results are as follows: 1. We show for every 1 ≤ p < 2, any oblivious subspace embedding with dimension r has distortion κ = 1 Ω (d1)1/p·log2/p r+(nr )1/p−1/2 . When r = poly(d) n in applications, this gives a κ = Ω(d1/p log−2/p d) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for p = 1 and the oblivious subspace embedding of Meng and Mahoney (STOC, 2013) for 1 < p < 2 are optimal up to poly(log(d)) factors. 2. We give sparse oblivious subspace embeddings for every 1 ≤ p < 2 which are optimal in dimension and distortion, up to poly(log d) factors. Importantly for p = 1, we achieve r = O(dlog d), κ = O(dlog d) and s = O(log d) non-zero entries per column. The best previous construction with s ≤ poly(log d) is due to Woodruff and Zhang (COLT, 2013), giving κ = Ω(d2poly(log d)) or κ = Ω(d3/2 log n· poly(log d)) and r ≥ d · poly(log d); in contrast our r = O(dlog d) and κ = O(dlog d) are optimal up to poly(log(d)) factors even for dense matrices. We also give (1) nearly-optimal `p oblivious subspace embeddings with an expected 1 + ε number of non-zero entries per column for arbitrarily small ε > 0, and (2) the first oblivious subspace embeddings for 1 ≤ p < 2 with O(1)-distortion and dimension independent of n. Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise `p low rank approximation. Our results give improved algorithms for these applications.