Load Frequency Control of Power Systems with an Energy Storage System Based on Safety Reinforcement Learning

Abstract

Load frequency control (LFC) is a critical component in power systems that is employed to stabilize frequency fluctuations and ensure power quality. As energy storage systems (ESSs) are increasingly integrated into the grid, managing additional constraints has become more challenging. To address these challenges, this paper proposes a safety reinforcement learning-based approach that incorporates ESSs into the LFC framework. By formulating a constrained Markov decision process (CMDP), this approach overcomes the limitations of conventional Markov decision processes (MDPs) by explicitly handling system constraints. Furthermore, a long short-term memory (LSTM)-based cost prediction critic network is introduced to improve the accuracy of cost predictions, and a primal-dual deep deterministic policy gradient (PD-DDPG) algorithm is employed to solve the CMDP. Simulation results demonstrate significant improvements: a 58.2% faster settling time, a 72.5% reduction in peak frequency deviation, and a 68.2% lower mean absolute error while maintaining all operational constraints.