ODD and Behavior-Based Approach to Scenario Coverage for Automated Driving Systems Testing

Abstract

The safe deployment of Automated Driving Systems (ADS) depends on rigorous safety assurance testing. Scenario-based testing has emerged as a key methodology, targeting edge cases and safety-critical situations that ADS may encounter. While previous research has primarily focused on defining these situations, known as scenarios, a significant gap remains in evaluating the completeness of the scenario sets used for testing. This work addresses that gap by introducing a structured, multi-level coverage framework designed to systematically assess whether a given scenario set comprehensively tests the conditions within the system's Operational Design Domain (ODD). The framework defines four coverage metrics: 1) scenario attribute range; 2) ODD and behaviour competency coverage; 3) out-of-ODD cases; and 4) rules of the road compliance. The proposed methodology leverages a modular simulation framework with an integrated analysis engine to facilitate attribute range exploration. It evaluates the test suite against the system's ODD attributes, including relevant combinations; tests the system in out-of-ODD scenarios; and applies scenario-rule matching techniques to assess compliance with rules of the road. The proposed framework's effectiveness is demonstrated through a case study involving an Automated Lane Keeping System (ALKS). Results highlight its ability to identify safety-critical scenarios, verify ODD coverage, assess out-of-ODD responses, and ensure regulatory compliance. This work advances the analysis phase of the safety performance assurance process and supports the standardisation of ADS verification and validation, contributing to their safe and trustworthy deployment.