An Autonomic Workflow Management System for Global Grids


Mustafizur Rahman and Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) Laboratory
Department of Computer Science and Software Engineering
The University of Melbourne, Australia
{mmrahman,raj}@csse.unimelb.edu.au


Abstract

Workflow Management System is generally utilized
to define, manage and execute workflow applications
on Grid resources. However, the increasing scale
complexity, heterogeneity and dynamism of Grid
environment that includes networks, resources and
applications have made such workflow management
systems brittle, unmanageable and insecure. Auto-
nomic computing provides a holistic approach for the
design and development of systems/applications that
can adapt themselves to meet requirements of per-
formance, fault tolerance, reliability, security, etc.,
without manual intervention. Therefore, this research
aims to design and develop mechanisms for building
an autonomic workflow management system that will
incorporate the properties of autonomic computing
and exhibit the ability to reconfigure itself to the
changes in the Grid environment, discover, diagnose
and react to the disruptions of workflow execution, and
monitor and tune Grid resources automatically.
1. Introduction
Since many of the large-scale scientific applications
executed on present-day Grids are expressed as com-
plex scientific workflows [1], workflow management
has emerged as one of the most important Grid services
in past few years. Scientific workflows that are also
known as Grid workflows, can be defined as the
aggregation of grid application services which are
executed on distributed heterogeneous resources in a
well defined order to satisfy the specific requirements
of users. A Workflow Management System (WMS) [2]
is generally employed to define, manage and execute
these workflow applications on Grid resources.
However, the increasing scale complexity, hetero-
geneity and dynamism of Grid environment that
includes networks, resources and applications have
made such workflow management systems brittle,
unmanageable and insecure. Autonomic computing
provides a holistic approach for the design and devel-
opment of systems/applications that can adapt them-
selves to meet requirements of performance, fault
tolerance, reliability, security, etc., without manual
intervention. Therefore, this research aims to design
and develop mechanisms for autonomic workflow
management that will enable a workflow management
system to incorporate the properties of autonomic
computing and exhibit the ability to protect itself,
recover from faults, reconfigure as required by changes
in the environment, and always maintain its operations
at a near optimal performance.
1.1. Motivation and Problem Statement
In the current approaches to workflow scheduling,
there is no cooperation between the distributed work-
flow brokers [3][4]. As a result, the problem of con-
flicting schedules can occur. For example, consider a
Grid environment (as shown in Figure 1) that consists
of n number of resources and m number of workflow
brokers. Users submit their scientific applications to
the workflow brokers. These brokers generate the
schedules based on the resource information obtained
from the Grid Information Services (GIS). A schedule
is effectively defined as the mapping of a set of tasks in
the workflow to a set of available resources. However,
if Workflow Broker 1 and Workflow Broker 2 query
the GIS at the same time, they will get the similar
information about the resource availability pattern.
Based on this information, Workflow Broker 1 and
Workflow Broker 2 will generate the same mapping
for the tasks in their locally submitted workflows,
which will lead to conflicting schedules. Therefore,
both the system and the application will suffer from
degraded performance if the resources are heavily
loaded.
Other major drawback involved with current work-
flow scheduling is that the existing workflow brokers
rely on the centralized (refer to Figure 1) or semi-
centralized hierarchical resource information services
such as MDS-2,3,4 [5]. Current studies have shown