Staged Design of Water Distribution Networks: A Reinforcement Learning Approach †

Abstract

Effectively planning the design of a water distribution network for the long term is a challenging task for water utilities, mainly due to the deep uncertainty that characterizes some of its most important design parameters. In an effort to navigate this challenge, this work investigates the potential of reinforcement learning in the lifecycle design of water networks. To this end, a deep reinforcement learning agent was trained to identify a sequence of cost-effective interventions across multiple construction phases within a network’s lifecycle under both deterministic and uncertain conditions. Our approach was tested on a modified benchmark of the New York Tunnels problem with promising results. The agent achieved comparable performance with the baseline heuristic algorithm in the deterministic setting and devised a flexible design strategy when multiple future scenarios were considered. These preliminary findings highlight the potential of reinforcement learning in the lifecycle design of water networks and represent a step towards the integration of more adaptive planning approaches in the field.