Heuristics-oriented overtaking decision making for autonomous vehicles using reinforcement learning

Abstract

This study presents a three-lane highway overtaking strategy for an automated vehicle, which is based on a heuristic planning reinforcement learning algorithm. The proposed decision-making controller focuses on keeping the autonomous vehicle operating safely and efficiently. First, the modelling of the overtaking driving scenario is introduced and the reference approaches named intelligent driver model and minimise overall braking induced by lane changes are formulated. Second, the Dyna-H algorithm, which combines the modified Q-learning algorithm with a heuristic planning policy, is utilised for highway overtaking decision-making. Three different heuristic strategies are formulated to improve learning efficiency and compare performance. This algorithm is applied to determine the lane change and speed selection for an ego vehicle in the environment with uncertainties. Finally, the performance of Dyna-H is estimated in the autonomous overtaking scenario by comparing it with the reference and traditional learning methods. Furthermore, the Dyna-H-enabled decision-making strategies are validated and analysed in an open-sourcing driving dataset. Results prove that the proposed decision-making strategy could produce superior performance in convergence rate and control.