Toward massive query optimization in large-scale distributed stream systems

18Citations
Citations of this article
28Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

Existing distributed stream systems adopt a tightly-coupled communication paradigm and focus on fine-tuning of operator placements to achieve communication efficiency. This kind of approach is hard to scale (both to the nodes in the network and the users). In this paper, we propose a fundamentally different approach and present the design of a middleware for optimizing massive queries. Our approach takes the advantages of existing Publish/Subscribe systems (Pub/Sub) to achieve loosely-coupled communication and to "intelligently" exploit the sharing of communication among different queries. To fully exploit the capability of a Pub/Sub, we present a new query distribution algorithm, which can adaptively and rapidly (re)distribute the streaming queries at runtime to achieve both load balancing and low communication cost. Both the simulation studies and the prototype experiments executed on PlanetLab show the effectiveness of our techniques. © 2008 Springer Berlin Heidelberg.

Cite

CITATION STYLE

APA

Zhou, Y., Aberer, K., & Tan, K. L. (2008). Toward massive query optimization in large-scale distributed stream systems. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 5346 LNCS, pp. 326–345). https://doi.org/10.1007/978-3-540-89856-6_17

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free