Federated data bases for the development of an operational monitoring and forecasting system of the ocean: The THREDDS dataset merger

Abstract

During the last decade, operational monitoring and forecasting systems have been developed in all the European seas. The exchange of data and products and the development of services for a wide community of users pose some fundamental issues, whose solution has become a priority in integrated and GMES referring projects, such as the MERSEA European project. These projects aim to develop a European system for operational monitoring and forecasting on global and regional scales of ocean physics, bio-chemistry and ecosystems. GMES system and its operational projects need to federate resources and expertise coming from diverse organizations working on different Earth Sciences fields (e.g. satellite data processing, in situ observing systems, data management, ocean and ecosystem modeling, etc.). Therefore, it is required a Marine Information Management (MIM) system capable of facilitating the regular real-time exchange of high quality information, data and products. Moreover, MIM system must provide appropriate information for a wide range of external users both in real-time and delayed mode. In this paper an architecture based on the OPeNDAP/THREDDS technology is proposed as a solution for these operational systems. In this context, a catalog merging solution is introduced for the MIM system, which results in the design and development of the THREDDS Dataset Merger (TDM): a set of services meant to merge THREDDS Dataset Inventory Catalogs, so to achieve a unique catalog service for a whole database federation. TDM service merges distributed and autonomous THREDDS catalogs in order to work out a virtual merged catalog. The TDM service was extended in order to provide automatic catalogs synchronization. This service allows extending the pull-based TDM paradigm to support push-based applications. Some security issues are also considered.