Chapter 7
Data Management
Craig Thomson, Kostas Kavoussanakis, Mark Sawyer, George Beckett,
Michal Piotrowski, Mark Parsons, and Arthur Trew
Abstract Data management is an important area of Grid research. It is concerned
with the storage, access, translation and integration of data. The Data Management
Technical Area of the BEinGRID project was set up to analyse the data requirements
of pilot projects and to support them in using data management related middleware.
After identifying these requirements it also developed design patterns to provide a
guide to other businesses which may face similar problems in the future. As a further
aid to businesses interested in adopting the Grid, the technical area also extended
existing middleware to allow it to implement some of the identified design patterns.
7.1 Introduction
Data management is an important area of Grid research. It is concerned with the
storage, access, translation and integration of data. It hopes to answer questions
like:
• Where should I put my data?
• How should I get to it?
• How do I present my data in a way others will understand?
• How can I combine data from different places?
All of these questions are important to modern businesses. In many industries,
collaboration and the efficient flow of information between organisations is critical.
For example, just-in-time techniques [14] aim to improve the efficiency of a supply
chain and to do this effectively they need access to up to date information from
multiple organisations.
One of the most familiar definitions of “The Grid” is a Computational Grid [15]
in which multiple distributed computer systems calculate a common result. One of
the things which characterises Grid computing is the heterogeneity of the computing
resources used. These differences can be in the hardware, the software or both and
interaction between the different resources is required for a Grid to be usable. A sim-
ilar definition can be applied to data and Data Grids; data from multiple sources can
C. Thomson (

)
EPCC, The University of Edinburgh, James Clerk Maxwell Building, Mayfield Road, Edinburgh
EH9 3JZ, UK
e-mail: c.thomson@epcc.ed.ac.uk
T. Dimitrakos et al. (eds.), Service Oriented Infrastructures
and Cloud Service Platforms for the Enterprise,
DOI 10.1007/978-3-642-04086-3_7, © Springer-Verlag Berlin Heidelberg 2010
141

142 C. Thomson et al.
be combined or used to produce a desired result or effect. In the case of data, the
heterogeneity can extend to the storage format as well as the machine type and the
software. Considerations such as the structure of data or the query language (xml,
SQL, files) as well as the particular software product (Oracle, MySQL, etc.) are
important additional considerations for a Data Grid and make this a very rich and
complex problem area.
The Data Management Technical Area of the BEinGRID project was set up to
analyse the data requirements of a number of pilot projects termed Business Exper-
iments (BEs) and to support them in using data management related middleware. In
addition it analysed the problems the BEs faced and extracted the common require-
ments multiple BEs had. After identifying these problems and requirements it also
developed design patterns to provide a guide to other businesses which may face
similar problems in the future. As a further aid to businesses interested in adopting
the Grid, the Technical Area also extended existing middleware to allow it to imple-
ment some of the identified design patterns. The focus of these middleware modi-
fications was OGSA-DAI [10], a long-standing data access and integration middle-
ware currently developed at The University of Edinburgh as part of the OMII-UK
project [11].
This chapter highlights some of the results that have been obtained during the
course of the project. It begins by defining the common technical requirements.
Then we examine some of the most relevant common capabilities. These are the
attributes that a solution to one of the common technical requirements must have.
Within the discussion of common capabilities we will also describe particular design
patterns which can be applied to solve these problems effectively. Where an imple-
mentation is available for a particular capability or to solve a common requirement,
it will also be discussed.
The results produced by the Data Management Technical Area came from the
analysis of concrete BEs in business sectors. Not all of the experiments had a strong
interest in data management and therefore the conclusions we have drawn are based
on our experiences with a subset of the experiments.
7.2 The Overall Challenge
The prominence of data in the majority of the BEs shows how important informa-
tion is in a modern business environment. The sheer variety of uses of data was a
challenge in itself when it came to analysing the different requirements of the many
BEs. It is difficult to give one single view of the challenge for data management.
Instead we list below some of the important areas which are relevant to the business
areas the project has investigated. These are as follows:
• Data transfer
• Integration of data from different organisations
• Replication of data between organisations
• Heterogeneous data.