Improving the Validation of Automotive Self-Learning Systems through the Synergy of Scenario-Based Testing and Metamorphic Relations

Abstract

Numerous applications in our everyday life use artificial intelligence (AI) methods for speech and image recognition, as well as the recognition of human behavior. Especially the latter application represents an interesting research field for self-learning systems based on AI methods in the automotive domain. Human driving behavior is determined by routines that an AI system can learn, thereby predicting future actions. However, the methods and tools for validating these systems are insufficient and need to be adapted to the new types of self-learning algorithms. Our framework combines scenario-based testing and metamorphic testing to address the challenges of ensuring correctness and reliability in dynamic and probabilistic SLS. A proof of concept is performed using the example of a self-learning comfort function in a vehicle. The correct functionality is shown by comparing the generated test cases. The concept addresses the main challenges in testing self-learning systems, in particular, the generation of test inputs and the creation of a test oracle.