Distinct estimate of set expressions over sliding windows

Abstract

Recently, lots of work focus on devising one-pass algorithms for processing and querying multiple data streams, such as network monitoring, sensor networks, etc. Estimating the cardinality of set expressions over streams is perhaps one of the most fundamental problems. Unfortunately, no solution has been devised for this issue over sliding windows. In this paper, we propose a space-efficient algorithmic solution to estimate the cardinality of set expression over sliding windows. Our probabilistic method is based on a new hash based synopsis, termed improved 2-level hash sketch. A thorough experimental evaluation has demonstrated that our methods can solve the problem efficiently. © Springer-Verlag Berlin Heidelberg 2005.