SCPOR: An elastic workflow scheduling algorithm for services computing

Abstract

Due to the complexity of scientific processes, computing and storage resources in a scientific workflow are often needed on an uneven basis, thus, the demand of resources is elastically changing during a run of a workflow. Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Resources assigned to a workflow cannot be automatically determined on demand of the size of the workflow and are not released to the environment until an execution of the workflow completes. The salient features of service-oriented computing have brought a new opportunity to schedule workflows on resources with elastically changing demand, as they allow resources to scale on demand as usage changes through dynamic provisioning. To address this issue, we firstly formalize a model of a service-oriented computing environment and a workflow graph representation for the environment. Then, we propose SCPOR, a scientific workflow scheduling algorithm that is able to schedule workflows in need of elastically changing compute resources. Our extensive experiments and comparisons for compute-intensive and data-intensive workflows have shown that SCPOR not only outperforms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically during workflow execution. © 2011 IEEE.