Sketching streaming histogram elements using multiple weighted factors

Abstract

We propose a novel sketching approach for streaming data that, even with limited computing resources, enables processing high volume and high velocity data eiciently. Our approach accounts for the fact that a stream of data is generally dynamic, with the underlying distribution possibly changing all the time. Speciically, we propose a hashing (sketching) technique that is able to automatically estimate a histogram from a stream of data by using a model with adaptive coeicients. Such a model is necessary to enable the preservation of histogram similarities, following the varying weight/importance of the generated histograms. To address the dynamic properties of data streams, we develop a novel algorithm that can sketch the histograms from a data stream using multiple weighted factors. The results from our extensive experiments on both synthetic and real-world datasets show the efectiveness and the eiciency of the proposed method.