Performance Evaluation Method for Automated Driving System in Logical Scenario

Abstract

With the continuous improvement of automated driving technology, how to evaluate the performance of an automated driving system is attracting more and more attention. Meanwhile, with the creation of scenario-based test methods, the traditional evaluation index based on a single test can no longer meet the requirements of high-level safety verification for automated driving system, and the performance evaluation of such a system in logical scenarios will be the mainstream. Based on the scenario-based test method and Turing test theory, a performance evaluation method for an automated driving system in the whole parameter space of a logical scenario is proposed. The logical scenario parameter space is partitioned according to the risk degree of concrete scenario, and the evaluation process in different zones are determined. Subsequently, the anthropomorphic index in the safe zone and the collision-avoidance index in the danger zone are defined by comparing test results of human driving and ideal vehicle motion. Taking front vehicle low-speed and cut-out scenarios as examples, two automated driving algorithms are tested in the virtual environment, and the test results are evaluated both by the proposed method and by human observation. The results show that the results of the proposed method are consistent with the subjective feelings of humans; additionally, it can be applied to scenario-based tests and the verification process of an automated driving system.