Testing autonomous and highly configurable systems: Challenges and feasible solutions

Abstract

Proving techniques and methods for safety critical systems in order to ensure a certain behavior as well as their corresponding safety requirements has still been a challenge for many years. Although the current situation in many areas like the automotive industry has improved a lot, new challenges are in sight especially when considering autonomous and adaptive systems approaching. Such systems have to reason about the current state and stimuli from their environment without humans in the loop or are allowed to change their behavior over time. Such systems induce new requirements for quality assurance and in particular testing. Here the focus has to be on providing guarantees of a wanted behavior before deployment of the systems even in case of changes or failures that might arise at runtime. In this paper, we discuss the underlying challenges and potential feasible solutions. In addition, we highlight similarities and differences with the current situation of testing safety critical systems.