Test automation for safety-critical systems: Industrial application and future developments

Abstract

Design, execution and evaluation of tests for safety-critical systems require considerable effort and skill and consume a large part of today's development costs. Due to the growing complexity of control systems, it has to be expected that their trustworthy test will become unmanageable in the future, if only conventional techniques, requiring a high degree of human interaction during the test process, are applied. In this article, we will focus on test automation for reactive real-time systems, with emphasis on Hardware-in-the-Loop tests analyzing the behaviour of combined software and hardware components. To illustrate possible approaches for this test problem, we describe a concept based on specifications written in Real-Time CSP. For the implementation of test generation and evaluation algorithms transition system representations are used, as can be obtained by Formal Systems' FDR tool. An industrial application of the method is presented and used for the evaluation of the benefits of formal methods-based testing in comparison with conventional techniques. Furthermore, we will indicate research topics in this field which are likely to become important for further improvements of the test process. Specifically, the benefits arising from an approach combining formal verification and testing will be discussed. Our presentation aims less at “promoting” a specific solution, but tries to illustrate the basic problems to be tackled with any formal method, when trying to develop test automation concepts to be applied in the context of reactive systems.