HydroRTC: A web-based data transfer and communication library for collaborative data processing and sharing in the hydrological domain

Abstract

The exponential growth in data generated by satellites, radars, sensors, and analysis and reanalysis from model outputs for the hydrological domain requires efficient real-time data management and distribution mechanisms. This paper introduces HydroRTC, a web-based data transfer and communication library designed to accelerate large-scale data sharing and analysis. Leveraging next-generation web technologies like WebSockets, WebRTC and Node.js, the library enables seamless peer-to-peer sharing, smart data transmission, and large dataset streaming. Three primary scenarios are presented as use cases, demonstrating the potential of HydroRTC as server-to-peer with intelligent data scheduling and large data streaming, peer-to-peer data sharing, and peer-to-server for data exchange. HydroRTC offers a promising solution for collaborative infrastructures in the hydrological and environmental domain, allowing real-time and high-throughput data sharing and transfer for enhancing research efficiency and collaboration capabilities.