Physics Constrained Reinforcement Learning for Improved Control of Transient Maneuvers in Nuclear Power Plants

Abstract

Operating nuclear power plants in a more flexible manner will be imperative to their success on future energy grids dominated by variable renewable energy generation. Load following of electrical power demands greater than 5%/minute in nuclear power stations is often impacted by safety limits imposed on components during the transient. The paper introduces a potential control approach to augment existing controllers in nuclear power plants through decoupling state variables from power during aggressive load following using a reinforcement learning agent. A test case for a nominal balance of plant (BOP) is proposed with an aggressive thermal load follow of electrical power at a rate of 15%/minute. The BOP dynamics are fully modelled in the Modelica language to achieve a high-fidelity picture of thermodynamic states in the system. A deep reinforcement learning model is implemented in the control feedback loop to pass a decoupling feedforward signal to the existing controller with the aim of reducing temperature deviation at the steam generator exit. It is shown that the reinforcement learning agent can improve the temperature deviations during the transient from a maximal deviation of 7.5°C to a maximal deviation of 1.5°C whilst also significantly shortening the duration the temperature is altered from its operating design point. The work also shows how this can be implemented without significantly altering other BOP dynamics during the transient. It is hoped that this research may contribute to improved economics of load following in plants by improving safety and reducing component degradation during ramping operation.