Management of real-time streaming data Grid services

Abstract

We discuss our message-based approach to managing real-time data streams and building higher level services to produce and consume them. Our messaging system acts as a substrate that can be used to provide qualities of service to various streaming applications ranging from audio-video collaboration systems to sensor Grids. The messaging substrates are composed of distributed, hierarchically arranged message broker networks. Services such as filters are deployed along the edges of the network. We discuss the role of management systems for both broker networks and filter services: broker network topologies must be created and maintained, and distributed filters must be arranged in appropriate sequences. These managed broker networks may be applied to a wide range of problems. We discuss applications to audio-video collaboration in some detail and also describe applications to streaming Global Positioning System data streams. These provide specific application filters that can transform and republish message streams to the broker system. Copyright © 2006 John Wiley & Sons, Ltd.