P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning

Abstract

Peer-to-peer (P2P) energy trading is a decentralized energy market where local energy prosumers act as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling and assume users' sustained active participation and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this article, we propose an automated P2P energy-trading framework that specifically considers the users' perception by exploiting prospect theory. We formalize an optimization problem that maximizes the buyers' perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy (DEbATE) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers' profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity (PQR), is based on Q-learning. Additionally, given the scalability issues of PQR, we propose a Deep Q-Network-based algorithm called ProDQN that exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve 26% higher perceived value for buyers and generate 7% more reward for sellers, compared to recent state-of-the-art approaches.