Higher order mutation testing to drive development of new test cases: An empirical comparison of three strategies

Abstract

Mutation testing, which includes first order mutation (FOM) testing and higher order mutation (HOM) testing, appeared as a powerful and effective technique to evaluate the quality of test suites. The live mutants, which cannot be killed by the given test suite, make up a significant part of generated mutants and may drive the development of new test cases. Generating live higher order mutants (HOMs) able to drive development of new test cases is considered in this paper. We apply multi-objective optimization algorithms based on our proposed objectives and fitness functions to generate higher order mutants using three strategies: HOMT1 (HOMs generated from all first order mutants), HOMT2 (HOMs generated from killed first order mutants) and HOMT3 (HOMs generated from not-easy-to-kill first order mutants). We then use mutation score indicator to evaluate, which of the three approaches is better suited to drive development of new test cases and, as a result, to improve the software quality.