Grid-based metadata services

Abstract

Data sets being managed in Grid environments today are growing at a rapid rate, expected to reach 100s of Petabytes in the near future. Managing such large data sets poses challenges for efficient data access, data publication and data discovery. In this paper we focus on the data publication and discovery process through the use of descriptive metadata. This metadata describe the properties of individual data items and collections. We discuss issues of metadata services in service rich environments, such as the Grid. We describe the requirements and the architecture for such services in the context of Grid and the available Grid services. We present a data model that can capture the complexity of the data publication and discovery process. Based on that model we identify a set of interfaces and operations that need to be provided to support metadata management. We present a particular implementation of a Grid metadata service, basing it on existing Grid services technologies. Finally we examine alternative implementations of that service.