Energy Management Strategy for Hybrid Electric Vehicles Based on Experience-Pool-Optimized Deep Reinforcement Learning

Abstract

The energy management strategy of Hybrid Electric Vehicles (HEVs) plays a key role in improving fuel economy and reducing battery energy consumption. This paper proposes a Deep Reinforcement Learning-based energy management strategy optimized by the experience pool (P-HER-DDPG), aimed at improving the fuel efficiency of HEVs while accelerating the training speed. The method integrates the mechanisms of Prioritized Experience Replay (PER) and Hindsight Experience Replay (HER) to address the reward sparsity and slow convergence issues faced by the traditional Deep Deterministic Policy Gradient (DDPG) algorithm when handling continuous action spaces. Under various standard driving cycles, the P-HER-DDPG strategy outperforms the traditional DDPG strategy, achieving an average fuel economy improvement of 5.85%, with a maximum increase of 8.69%. Compared to the DQN strategy, it achieves an average improvement of 12.84%. In terms of training convergence, the P-HER-DDPG strategy converges in 140 episodes, 17.65% faster than DDPG and 24.32% faster than DQN. Additionally, the strategy demonstrates more stable State of Charge (SOC) control, effectively mitigating the risks of battery overcharging and deep discharging. Simulation results show that P-HER-DDPG can enhance fuel economy and training efficiency, offering an extended solution in the field of energy management strategies.