A workflow model for heterogeneous computing environments

  • Curran O
  • Shearer A
  • 1

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract

This work presents the details of a model for fully decentralised scientific workflow management designed to support the efficient execution of applications in heterogeneous and unpredictable computing environments. The proposed model involves a novel application of the overlay metacomputer concept, built on a graph-oriented peer to peer framework. The use of peer to peer overlay on both batch and cycle scavenging systems simultaneously enables the aggregation and virtual homogenisation of heterogeneous resource collectives, and makes possible the execution of complex scientific workflow applications on otherwise batch oriented processors. Overlaying a peer to peer workflow management system on multiple independent sites decouples task expression from resource allocation, i.e., the host resource managers allocate processing elements to host the peers, allowing the peers execute multiple workflow tasks. This facilitates the efficient execution of very fine grained workflow expressions, which is shown to reduce both application makespan and wasted cycles when using processing elements allocated by a pre-emptive resource manager. Quantitative results are presented based on performance observed in a large scale heterogeneous multi-user environment.

Author-supplied keywords

  • Condensed graphs
  • Fine grained workflow
  • Heterogeneous resource aggregation
  • Pre-emptive environments
  • Webcom

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Oisín Curran

  • Andy Shearer

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free