Active Power Correction Strategies Based on Deep Reinforcement Learning - Part I: A Simulation-driven Solution for Robustness

Abstract

Employing the novel Deep Reinforcement Learning approach, this paper addresses the active power corrective control in modern power systems. Seeking to minimize the joint effect engendered by operation cost and blackout penalty, this correction strategy focuses on evaluating the robustness and adaptability aspects of the control agent. In Part I of this paper, where robustness is the primary focus, the agent is developed to handle unexpected incidents and guide the stable operation of power grids A Simulation-driven Graph Attention Reinforcement Learning method is proposed to perform robust active power corrective control. The aim of the graph attention networks is to determine the representation of power system states considering the topological features. Monte Carlo tree search is adopted to select the best suitable action set out of the large action space, including generator redispatch and topology control actions. Finally, driven by simulation, a guided training mechanism along with a long-short-term action deployment strategy are designed to help the agent better evaluate the action set while training and to operate more stably when deployed. The efficacy of the proposed method has been demonstrated in the '2020 Learning to Run a Power Network - Neurips Track 1' global competition and the associated cases. Part II of this paper deals with the adaptability case, where the agent is equipped to better adapt to a grid that has an increasing share of renewable energies through the years.