Energy Management for Integrated Energy System Based on Coordinated Optimization of Electric–Thermal Multi-Energy Retention and Reinforcement Learning

Abstract

With the large-scale access to a large number of distributed electric and thermal flexible resources and multiple loads on the user side, the energy management of the integrated energy system (IES) has become an effective way for the efficient and low-carbon economic operation of energy systems. In order to explore a new mode of IES energy management with the participation of energy service providers (ESPs) and user clusters (UCs), this paper puts forward an energy management method for electric–thermal microgrids, considering the optimization of user energy consumption characteristics. Firstly, an energy management framework with multi-agent participation of ESP and user cluster is proposed, and a user energy preference model is established considering the user’s electricity and heat consumption preferences. Secondly, considering the operation benefit of ESP and user cluster, based on the reinforcement learning (RL) framework, an energy management model between ESPs and users is established, and a distributed solution algorithm combining Q-learning and quadratic programming is proposed. Finally, the IESs with different user scales and energy units are taken as the test system, and the optimal energy management strategy of the system, considering the user’s energy preference, is analyzed. The simulation results demonstrate that the energy management model proposed enhances the economic efficiency of IES operations and reduces emissions. In a test system with two UCs, the optimized system achieves a 5.05% reduction in carbon emissions. The RL-based distributed solution algorithm efficiently solves the energy management model for systems with varying UC scales, requiring only 6.55 s for systems with two UCs and 13.26 s for systems with six UCs.