An adaptive meta-scheduler for data-intensive applications

Abstract

In data-intensive applications, such as high-energy physics, bioinformatics, we encounter applications involving numerous jobs that access and generate large datasets. Effective scheduling such applications is challenging, due to a need to consider for both computational resources and data storage resources. In this paper, we describe an adaptive scheduling model that consider availability of computational, storage and network resources. Based on this model we implement a scheduler used in our campus grid. The results achieved by our scheduler have been analyzed by comparing Greedy algorithm that is widely used in computational grids and some data grids. © Springer-Verlag 2004.