Deep Reinforcement Learning Based Pricing Strategy of Aggregators Considering Renewable Energy

Abstract

With the rapid development of information and communications technology and high penetration of renewable energy, the role of an aggregator in a smart grid has emerged to better coordinate power and cash flows between energy producers and consumers through the adjustment of pricing signals. This study proposes variation indices about the statistics of renewables and a control law for an energy storage system. A deep reinforcement learning based pricing strategy of an aggregator for profit maximization in consideration of the energy balance is developed accordingly. The proposed approach can consider opponents' behaviors, variability of renewables, and varying bounds of charging and discharging events in a nonstationary environment, which can be hardly addressed by pricing strategies based on conventional learning algorithms such as Q-learning and deep Q-network. Numerical analysis using real-world data shows that the proposed approach can outperform existing pricing strategies in terms of the learning speed and profit of aggregators.