Extended covering arrays for sequence coverage

Abstract

Although combinatorial testing has been widely studied and used, there are still some situations and requirements that combinatorial testing does not apply to well, such as a system under test whose test cases need to be performed contiguously. For thorough testing, the testing requirements are not only to cover all the interactions among factors but also to cover all the value sequences of every factor. Generally, systems under test always involve constraints and dependencies in or among test cases. The constraints among test cases have not been effectively specified. First, we introduce extended covering arrays that can achieve both t-way combinatorial coverage and t-wise sequence coverage, and propose a clocked computation tree logic-based formal specification method for specifying constraints. Then, Particle Swarm Optimization (PSO) based Extended covering array Generator (PEG) is elaborated. To evaluate the constructed test suites, a method for verifying the constraints' validity is presented, and kernel functions for measuring the coverage are also introduced. Finally, the performance of the proposed PEG is evaluated using several sets of benchmark experiments for some common constraints, and the feasibility and usefulness of PEG is validated.