Swift2: High performance software for short-medium term ensemble streamflow forecasting research and operations

Abstract

The second version of the Short-term Water Information and Forecasting Tools (SWIFT2) is a suite of tools for flood and short-term streamflow forecasting, consisting of a collection of hydrologic model components and utilities. It is currently developed within the Water Information Research and Development Alliance, with a requirement to address both research and operational modelling needs. Catchments are modelled using a traditional approach for semi-distributed models: conceptual subareas and a node-link structure for channel routing. SWIFT2 comprises modules for calibration, model state updating, ensemble runs, output error correction and data assimilation. While applicable to simpler modelling tasks, SWIFT2 is designed from the ground up to tackle computationally intensive problems in probabilistic forecasting research, with sub-daily time steps over decades, and ensemble sizes in the thousands. Typical retrospective ensemble simulation outputs can reach hundreds of gigabytes, and projected runtimes can reach days if not parallelized. SWIFT2 simulations are safe for in-memory copy and parallel executions, can make effective use of multi-core personal computers or high-performance compute clusters to address computationally demanding research or operational requirements. SWIFT2 is written in modern C++ and is platform-agnostic. SWIFT2 is architected to offer several options for user interfaces. Driven by the needs of current users, it is seamlessly accessible from high-level interactive languages including R, Python and MATLAB (Figure 1). To achieve this, the SWIFT API (Application Programming Interface) offers a controlled yet fully fledged access to core modelling features and concepts. The time series handling is performed in a separate library for reuse outside of SWIFT2, using advanced C++ templates, similar to the Boost libraries. Users have access via concise commands in their interactive language of choice to high level modelling concepts, such as simulations, parameterizers, state initializers and objective calculators. SWIFT2 already offers sophisticated features for calibration such as composability of parameter sets, definition of meta-parameters and non-trivial hard constraints on parameter feasibility. This permits the formulation of modelling goals, e.g. calibration, for use in conjunction with state of the art third-party model analysis tools such as metaheuristics frameworks. While the robustness and correctness of SWIFT2 are the primary development goals, runtime performance and parallel scalability are important. Initial measures and usages already show amply adequate operation in calibration mode on multi-core computers, with a very good multi-threaded parallel scalability on larger catchment models. Known performance tuning strategies have been investigated and will further enhance the raw computational performance.