Physically Consistent Runoff Simulation in Mountainous Catchments Using a Time-Varying Gated Hybrid XAJ–LSTM Model

Abstract

Accurate simulation of rainfall–runoff processes in mountainous catchments is essential for flood forecasting and water resource management. Traditional physically based models often suffer from structural rigidity and parameter uncertainty, while deep learning models, although effective in capturing nonlinear patterns, lack physical constraints and interpretability. To address these issues, this study developed a time-varying gated hybrid model (XAJ–LSTM) that integrates the Xinanjiang (XAJ) model with a Long Short-Term Memory (LSTM) network to improve runoff prediction accuracy and physical consistency. Hourly rainfall, temperature, potential evapotranspiration, and runoff data (2015–2023) from 17 small to medium mountainous catchments in Shi Yan and En Shi, Hubei Province, were used to drive and evaluate the XAJ, LSTM, and XAJ–LSTM models. The hybrid model achieved mean NSE and KGE values of 0.971 ± 0.020 and 0.962 ± 0.024, respectively, outperforming both individual models. In about 80% of the catchments, the gating parameter λ(t) showed a negative correlation with discharge, indicating adaptive adjustment between the physical and data-driven components. The coupled model reproduced both high- and low-flow processes well, with deviations in flow duration curves generally within ±5%. These findings demonstrate that the proposed time-varying gating structure effectively balances model accuracy, stability, and interpretability.