Hierarchical energy optimization of flywheel energy storage array systems for wind farms based on deep reinforcement learning

Abstract

Due to the volatility and intermittency of renewable energy, injecting large amounts of renewable energy into the grid will have a tremendous impact on the stability and security of the network. In this paper, we propose the hierarchical energy optimization of flywheel energy storage array system (FESAS) applied to smooth the power output of wind farms to realize source-grid-storage intelligent dispatching. The energy dispatching problem of the FESAS is described as a Markov decision process by the actor-critic (AC) algorithm. In order to solve the problems of stability and low sampling efficiency of the AC algorithm, the soft actor-critic (SAC) algorithm, a deep reinforcement learning (DRL) algorithm based on the model-free off-policy method of the maximum entropy framework, is adopted. Furthermore, SAC and prioritized experience replay (PER) are utilized to greatly improve learning efficiency and sample utilization. The experimental results show that SAC-PER has better performance and stability in energy optimization of the FESAS.