Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends

Abstract

The proper functioning of connected and autonomous vehicles (CAVs) is crucial for thesafety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomousdriving requires a long period of mixed autonomy traffic, including both CAVs andhuman-driven vehicles. Thus, collaborative decision-making technology for CAVs is essential togenerate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomytraffic. In recent years, deep reinforcement learning (DRL) methods have become an efficient way insolving decision-making problems. However, with the development of computing technology, graphreinforcement learning (GRL) methods have gradually demonstrated the large potential to furtherimprove the decision-making performance of CAVs, especially in the area of accurately representingthe mutual effects of vehicles and modeling dynamic traffic environments. To facilitate the developmentof GRL-based methods for autonomous driving, this paper proposes a review of GRL-basedmethods for the decision-making technologies of CAVs. Firstly, a generic GRL framework is proposedin the beginning to gain an overall understanding of the decision-making technology. Then, theGRL-based decision-making technologies are reviewed from the perspective of the constructionmethods of mixed autonomy traffic, methods for graph representation of the driving environment,and related works about graph neural networks (GNN) and DRL in the field of decision-makingfor autonomous driving. Moreover, validation methods are summarized to provide an efficientway to verify the performance of decision-making methods. Finally, challenges and future researchdirections of GRL-based decision-making methods are summarized.