Self-Learning Enhanced Energy Management for Plug-in Hybrid Electric Bus with a Target Preview Based SOC Plan Method

Abstract

This paper proposes a self-learning enhanced energy management for plug-in hybrid electric bus (PHEB). Two innovations are made to distinguish our contributions from others. First, a target preview-based reference SOC plan method is proposed, where a linear reference SOC trajectory will be re-planed at fixed time steps (every 60 s) by taking the expected SOC at the destination as the target, and a feasible zone is exclusively defined to provide a margin for the fuel economy improvement. Second, a reinforcement learning-Pontryagin's minimum principle (RL-PMP)-based energy management constituted by the RL and PMP algorithms is proposed, where the average velocity with time moving, the SOC, and the normalized trip distance are taken as states and the co-state is taken as the action. Different from the conventional RL methods, the co-sate is only updated at the fixed time step, and the reward will be evaluated in the next 60 s. This can greatly accelerate the training process of the RL-PMP. In addition, the driving conditions in the next 60 s are predicted by a designed Markov chain. The simulation results demonstrate that the fuel economy of the RL-PMP is close to the dynamic programming (DP) and can be averagely improved by 14.155% compared to the rule-based control strategy.