Precision vs confidence tradeoffs for ℓ2-based frequency estimation in data streams

Abstract

We consider the data stream model where an n-dimensional vector x is updated coordinate-wise by a stream of updates. The frequency estimation problem is to process the stream in a single pass and using small memory such that an estimate for xi for any i can be retrieved. We present the first algorithms for ℓ2-based frequency estimation that exhibit a tradeoff between the precision (additive error) of its estimate and the confidence on that estimate, for a range of parameter values. We show that our algorithms are optimal for a range of parameters for the class of matrix algorithms, namely, those whose state corresponding to a vector x can be represented as Ax for some m x n matrix A. All known algorithms for ℓ2-based frequency estimation are matrix algorithms. © Springer-Verlag 2012.