Exploiting symmetries to test programs

Abstract

Symmetries often appear as properties of many artifical settings. In program testing, they can be viewed as properties of programs and can be given by the tester to check the correctness of the computed outcome. In this paper, we consider symmetries to be permutation relations between program executions and use them to automate the testing process. We introduce a software testing paradigm called symmetric testing, where automatic test data generation is coupled with symmetries checking to uncover faults inside the programs. A practical procedure for checking that a program satisfies a given symmetry relation is described. The paradigm makes use of group theoretic results as a formal basis to minimize the number of outcome comparisons required by the method. This approach appears to be of particular interest for programs for which neither an oracle, nor any formal specification is available. We implemented symmetric testing by using the primitive operations of the Java unit testing tool Roast by N. Daley, D. Hoffman, and P. Strooper (2002). The experimental results we got on faulty versions of classical programs of the software testing community tend to show the effectiveness of the approach.