Energy Management of a Plug-In Hybrid Electric Vehicle Using Bayesian Optimization and Soft Actor-Critic Algorithm

Abstract

The emerging deep reinforcement learning (DRL) algorithms are promising for energy management of plug-in hybrid electric vehicles (PHEVs), but most existing DRL methods cannot meet the requirement for learning robustness and data independence, which are essential to mission-critical systems, such as PHEVs. By incorporating Bayesian optimization (BO) with the soft actor-critic (SAC) algorithm, this article proposes a new energy management strategy (EMS) for HEVs, namely, BO-SAC EMS. This work contributes solutions to address the abovementioned two challenges in DRL-based EMS: 1) through BO-based hyperparameter tuning, the brittle convergence characteristics and robustness of the SAC algorithm have been significantly improved and 2) by introducing a state-action-reward (SAR) codesign scheme for the SAC algorithm, the dependence on real-world data has been considerably reduced, thus improving training efficiency. Using the original SAC (Origin-SAC) method and dynamic programming (DP) results as the baseline, comparison studies are conducted under ten driving cycles. By estimating the maximum energy consumption per 100 km based on the Six Sigma theory, BO-SAC is shown more robust by saving more than 3% energy in the worst case.