Intersection decision model based on state adversarial deep reinforcement learning

Abstract

The decision-making model for autonomous vehicles based on deep reinforcement learning faces challenges in robustness in complex and dynamic scenarios, such as crossroads. This study proposes a state adversarial deep reinforcement learning decision model for autonomous driving to address this concern. The model incorporates an adversarial state space standard deviation to accommodate the significant differences in the decision model’s state space between the initial and final steps. Adversarial state space standard deviation, coupled with real-time state space, creates an adversarial state space set. The decision model obtains an adversarial strategy under the adversarial state space set and integrates it with the strategy of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. This simultaneous optimization of model parameters is designed to improve the robustness of the decision model for autonomous driving. Experimental validation was performed using the Carla simulation environment, involving the creation of four scenarios with varying traffic flow densities and collision rates. In these scenarios, the enhanced model achieved success rates ranging from 89.9% to 98.6%, with the average vehicle travel time staying below 10 s. By contrast, the TD3 and Soft Actor-Critic (SAC) models exhibited success rates between 78.4% and 93.4%, with average vehicle travel times ranging from 10.03 to 20.01 s. When tested across various scenarios, the improved model demonstrated greater robustness compared to the TD3 and SAC models.