GEDAS: A Data Management System for Data Grid Environments

Abstract

In data grid environments, many large-scale scientific experiments and simulations generate very large amounts of data in the distributed storages, spanning thousands of files and data sets. In such environments, the replication technique for the fast data sharing between the community of researchers, and the high-performance I/O for the storage and efficient data accesses on heterogeneous resources present an extremely challenging task. Several data replication techniques have been developed to support high-performance data accesses to the remotely produced scientific data. However, most of those techniques were implemented with the assumption that the data being replicated is read-only so that it would not be modified once it has been generated. Furthermore, those techniques mainly focus on measuring up the network performance, but ignoring I/O overhead incurred during the data generation and replication. We have developed a software system, called Grid Environment-based Data Management System (GEDAS), that provides a high-level, user-friendly interface, while maintaining the consistent data replicas among the grid communities. We describe the design and implementation of GEDAS and present performance results on Linux cluster. © Springer-Verlag Berlin Heidelberg 2005.