Dissemination of models over timevarying data

Abstract

Dissemination of time-varying data is essential in many ap-plications, such as sensor networks, patient monitoring, stock tickers, etc. Often, the raw data have to go through some form of pre-processing, such as cleaning, smoothing, etc, be-fore being disseminated. Such pre-processing often applies mathematical or statistical models to transform the large volumes of raw, point-based data into a much smaller num-ber of piece-wise continuous functions. In such cases, the necessity to distribute data models instead of raw data may arise. Nevertheless, model dissemination has received very little attention so far. In this paper, we attempt to ll this gap and propose a model-agnostic dissemination framework that can handle different models in a uniform manner. The dissemination infrastructure is built on top of a tree-based overlay network, reminiscent to the ones employed in pub-lish/subscribe systems, which are known to scale well to the number of data producers and receivers. To adequately deal with the vast model variation and receivers' very different accuracy requirements on the models, we have developed optimized model routing algorithms, which are intended to minimize data trac and avoid bottlenecks within the dis-semination network. The extensive experimental evaluation over a prototype system that we have built shows that our methods are both effective and robust. © 2011 VLDB Endowment.