On-line grid monitoring based on distributed query processing

Abstract

Existing Grid monitoring approaches do not combine three desirable features: on-line access to monitoring data, advanced query capabilities and data reduction. We present a solution for on-line monitoring of large-scale computing infrastructures based on Complex Event Processing principles and technologies. We focus on leveraging CEP for distributed processing of client queries and monitoring data streams. This results in significant reduction of network traffic due to on-line monitoring. We discuss benefits of CEP-based approach to monitoring and describe details of processing queries in a distributed way. A case study - monitoring of load caused by jobs in a Grid infrastructure - is presented. Performance evaluation to investigate monitoring overhead in terms of CPU, memory and network traffic is also provided. © 2012 Springer-Verlag.