Real-time and hybrid systems testing

Abstract

In this chapter, we have discussed test case generation for real-time and hybrid systems. As we have seen in the beginning, test automation for real-time and hybrid systems differs from that for untimed systems. The time domain has to be taken into account as well as the dense value domain for hybrid systems, e.g. for test evaluation. The model of the SUT must abstract from the real behavior of the SUT to allow sensible test case generation. In contrast, the test driver concretizes the generated inputs to feed the SUT. Vice versa, the concrete outputs of the SUT are abstracted again so that test evaluation can be done with respect to the abstract model. Process Algebra vs. Automata Different modeling techniques can be used for this purpose. We have discussed ACSR as a process algebra and different variants of automata that already have been used for test case generation. A process algebra is capable of modeling a system consisting of processes that communicate with each other via events. Concurrency can be modeled explicitly as there are operators for interleaving and parallel execution with synchronization. Time is considered discrete. One problem of process algebras is that this modeling technique has no support in industry where graphical modeling is preferred. Nevertheless, testing based on process algebra has been proven useful and practicable. In contrast, automata based modeling has more support as this is very popular, e.g. with respect to UML where state machines are used as automata variants. Timed automata introduce either discrete or dense time to be capable of modeling timing constraints. Graphical models are in principle easy to understand as control flow can be captured at one sight. However, large models become intractable in graphical representation. Hierarchy or different abstraction levels must be used to better this situation. Discrete vs. Dense Time With respect to time, this is either modeled discrete or dense. In the first case, a timer is a counter that is incremented continuously. As the computer itself is working discrete, this seems appropriate. In contrast, time is naturally dense so modeling based on dense time is closer to reality, e.g. when analog sensors and actuators are used. For real-time systems, it must be chosen if the model represents natural time or computer time beforehand. For hybrid systems, time must be modeled dense as the continuous parts of the system rely on this. Discrete vs. Dense Values Another aspect is the value domain that has to be tested. This is considered discrete for real-time systems. We have seen that test case generation builds up on methods for untimed systems and enhances the generated test cases with timing information. Discrete values for inputs and outputs to respectively from the SUT can be easily generated and evaluated. Time points and intervals for generating inputs and outputs are chosen with respect to the selected test case generation method. For hybrid systems, this situation differs as we have dense-valued variables. Until now, nobody has tackled the problem of generating curves as input data if time-continuous input is needed. With respect to output data, fuzziness is considered for test evaluation. Test Case Generation Algorithms We presented two different test case generation algorithms for real-time systems that are based on discrete time and one algorithm that is based on dense time. Also one algorithm for creating test suites for hybrid systems was discussed. The most important function of these algorithms is the way in which the size of the test suite is reduced to a manageable size, i.e the way in which test cases are selected. The first test case generation algorithm discussed is based on the process algebra ACSR where test cases are derived from the constraint graph of the ACSR model. Test data is selected to cover two type of timing constraints, namely performance constraints and behavioral constraints. To handle also applications with recursive elements, upper bounds on the length of tested execution scenarios have to be introduced. Furthermore, test case generation for systems with discrete time can be done with timed automata models. Here, the central idea is using test views that restrict the model of the SUT to a specified test purpose. Test views are composed by parameters like clock granularity and the division of actions into observable and hidden actions. Therefore the task of the test engineer is creating test views that lead to manageable test suites. This must be done carefully as parts of the system may never be tested while other test views overlap. One important advantage of this approach is that the size of test suites is scalable. With respect to dense time, the main idea is finding equivalent parts in the model called partitions. Due to the dense time domain, the model has infinitely many states with respect to the infinitely many possible time values. Partitions group these values so that a finite graph is the result that can be used for test case generation. As this may be too large for full exploration, heuristics must be used to limit the size of the state space. For hybrid systems, the presented algorithm is based on the results of test case generation for real-time systems. In addition to the hybrid model, a second, discrete, model is created that abstracts from the hybrid system. This can be used in combination with the test case generation algorithms for real-time systems presented. The original hybrid model is needed to derive correct evaluations for variables as the discrete model being only an abstraction is not capable of doing this. The selection of dense input curves is not tackled. Optimization of Test Suites Testing timing constraints is still a difficult topic as delays for stimulating the SUT must be chosen and the correctness of outputs of the SUT must be accessed. First attempts with using evolutionary theory based test case generation methods have shown that these can improve the test suite with respect to determining best- and worst-case-execution times. This can be helpful to prove if a SUT meets its timing requirements. All other algorithms presented built up on results of testing untimed systems. In contrast, evolutionary testing has a different background and therefore provides a new point of view for future work in the field of real-time and hybrid systems testing. Future Work Model-based test case generation for real-time and hybrid systems has been successfully applied in relatively small examples. More effort has to be put into this to prove the practicability of test case generation algorithms. There is also only few tool support for test case generation and execution for real-time systems and no tool support for hybrid systems. New techniques like evolutionary testing to test timing constraints must be further surveyed. Also cross-fertilization between the different approaches seems useful to improve the presented algorithms as each has its advantages and disadvantages. © Springer-Verlag Berlin Heidelberg 2005.