Improving the Computational Efficiency of the Unit Commitment Problem in Hydrothermal Systems by Using Multi-Agent Deep Reinforcement Learning

Abstract

In power systems with a significant hydroelectric component, instances of the Unit Commitment (UC) problem may be much more computationally intensive due to the longer decision horizons and the additional hydro constraints. Therefore, this paper presents a methodology to reduce the solution space to accelerate 168-hour-ahead UC formulated as a Mixed-Integer Linear Program (MILP). First, an offline model maps environment observations to actions in a Multi-Agent Deep Reinforcement Learning (MADRL) model. This mapping uses historical power system operation data to determine the on/off status of specific generation units. Then, the online model uses the binary variable solutions obtained by the offline model to solve a UC problem with a reduced solution space. The Multi-Agent approach allows each agent, based on Artificial Neural Networks (ANN) with a Temporal Convolutional Network (TCN) architecture, to group units that are located in the same region. A shared cumulative reward function is used to adjust simultaneously the different ANN weights during the learning phase. The effectiveness of our method is demonstrated using real operational data of the Chilean National Electricity System, achieving statistically significant lower computation times and a negligible error that is within the integrality gap of the solver.