On strong mutation and the theory of subsuming logic-based mutants

Abstract

Redundant mutants might cause problems when benchmarking since testing techniques can get high scores without detecting any nonredundant mutants. However, removing nonredundant mutants might cause similar problems. Subsumed mutants are per definition also redundant since no additional tests are required to detect them once all other mutants are detected. We focus on relational operator replacement (ROR) and conditional operator replacement mutants. Subsumption relations between ROR mutants are defined by fault hierarchies. The fault hierarchies are proven for weak mutation but have since they were published been used with strong mutation. We prove that ROR fault hierarchies do not hold for strong mutation and show why. We also show that the probability for a random test to experience the problem can be more than 30% and that 50% of the mutants might be affected in a real software system. Finally, we show that there is a similar problem with the theory on sufficient conditional operator replacement.