An adaptive skeletal task farm for grids

Abstract

Algorithmic skeletons abstract commonly used patterns of parallel computation, communication, and interaction. By demonstrating a predictable communication and computation structure, they provide a foundation for performance modelling and estimation. Grids pose a challenge to known distributed systems techniques as a result of their dynamism. One of the most prominent research areas concerns the availability of proved programming paradigms with special emphasis on the performance side. Thus, adaptable performance improvement techniques have been the subject of intense scrutiny. Scant research has been conducted on using the skeletal predicting information to enhance performance in heterogeneous environments. We propose the use of these predicting properties to adaptively enhance the performance of skeletons, in particular of a task farm, within a computational grid. Hence, the problem addressed in this paper is: given a skeletal task farm, find an effective way to improve its performance on a heterogeneous distributed environment by incorporating information at compile time that helps it to adapt at execution time. This work provides a grid-enabled, adaptive task farm model, using the NWS statistical predictions on bandwidth, latency and processor availability. The central case study implements an ad-hoc task farm based on C/MPI and employs PACX-MPI for inter-node communication. We present initial promising results of parallel executions of an artificially-generated numerical code in a grid. © Springer-Verlag Berlin Heidelberg 2005.