Complexity Evaluation of Test Scenarios for Autonomous Vehicle Safety Validation Using Information Theory

Abstract

The validation of autonomous vehicles remains a vexing challenge for the automotive industry’s goal of fully autonomous driving. The systematic hierarchization of the test scenarios would provide valuable insights for the development, testing, and verification of autonomous vehicles, enabling nuanced performance evaluations based on scenario complexity. In this paper, an information entropy-based quantification method is proposed to evaluate the complexity of autonomous vehicle validation scenarios. The proposed method addresses the dynamic uncertainties within driving scenarios in a comprehensive way which includes the unpredictability of dynamic agents such as autonomous vehicles, human-driven vehicles, and pedestrians. The numerical complexity calculation of the approach and the ranking of the scenarios are presented through sample scenarios. To automate processes and assist with the calculations, a novel software tool with a user-friendly interface is developed. The performance of the approach is also evaluated through six example driving scenarios, then through extensive simulation using an open-source microscopic traffic simulator. The performance evaluation results confirm the numerical classification and demonstrate the method’s adaptability to diverse scenarios with a comparison of complexity calculation ranking to the ratio of collision, near collision, and normal operation tests observed during simulation testing. The proposed quantification method contributes to the improvement of autonomous vehicle validation procedures by addressing the multifaceted nature of scenario complexities. Beyond advancing the field of validation, the approach also aligns with the broad and active drive of the industry for the widespread deployment of fully autonomous driving.