Efficient processing of stream data over persistent data

Abstract

A data stream is a continuous sequence of items produced in real time. A stream can be considered to be a relational table of infinite size [1]. It is therefore considered impossible to maintain an order of the items in the stream with respect to an arbitrary attribute. Likewise, it is impossible to store the entire stream in memory. However, results of operations are expected to be produced as soon as possible. As a consequence, standard relational query processing cannot be straightforwardly applied, and online stream processing has become a new field of research in the area of data management. A number of common examples where online stream processing is important are network traffic monitoring [2–6], sensor data [7], web log analysis [8,9], online auctions [10], inventory and supply-chain analysis [11–13], as well as real-time data integration [14,15].