Coupling factor and cost based task clustering method to optimize task clustering for scientific workflows in cloud environment

Abstract

Scientific workflows are large scale loosely coupled submissions that are used by Computational Scientists. They are composed of multiple tasks with dependencies between them and are composed of many fine granular tasks. Task clustering is an optimization method that combines multiple tasks into a single job such that task execution time and system overhead is reduced and thus the whole performance is improved in a cloud environment. Though existing task clustering algorithms has significantly reduced the System overhead, yet dependencies among the tasks are not well-thought-out. This work examines the features of task by which the tasks can be clustered and developed proficient task clustering algorithm. In this work two task clustering ideas were proposed namely Horizontal Coupling Factor (HCF) based clustering and Horizontal Processing Cost (HPC) based Task Clustering. Next, the proposed algorithm have been evaluated and tested for various real world applications and the experiment results shows that the proposed approach suits best for data intensive and Compute intensive applications. The obtained results showed that the HCF and HPC task clustering strategies can significantly improve the performance by reducing the task execution time and inter task Communication delay.