Object-oriented testing capabilities and performance evaluation of the C# mutation system

Abstract

The main purpose of mutation testing approach is to check a test suite quality in terms of the adequacy to killing programs with inserted programming faults. We present advances in the C# mutation testing system that supports object-oriented mutation operators. The system enhancements related to functional requirements (mutation operators, avoiding generation of invalid and partially of equivalent mutants) and non-functional ones (speed-up using a new parser and reflection, space reduction storing mutant updates). Mutation testing of six widely used open source programs is discussed. The quality of the tests supplied with these programs was experimentally determined. Performance measures were evaluated to assess system enhancements (2-4 faster mutants creation, 10-100 times disk space reduction, tradeoff of time overhead for storing mutants of different size in a local or remote repository). © 2012 Springer-Verlag.