Possibility of cost reduction by mutant clustering according to the clustering scope

Abstract

Mutation testing offers developers a good way to improve the quality of a test set. However, the high cost of executing a large number of mutants remains an issue. This paper examines the possibility of reducing the cost of statement-level mutant clustering by comparing the number of mutant executions with those of expression-level and block-level mutant clustering. The goal is to investigate to what extent the clustering scope should be extended. The experimental results using nine real-world programs show that statement-level clustering can reduce the mutant executions that are required by expression-level clustering by 10.51% on average. Block-level clustering exhibits an unexpected result; the number of mutant executions with block-level clustering is only 1.06% times less than that with statement-level clustering. That is, statement-level clustering is more cost-effective than block-level clustering when considering their clustering overheads. A compound expression plays a major role in providing a cost-reduction effect in statement-level clustering. With a compound expression, the number of candidate mutants to be clustered in a statement scope increases, and state change can be comprehensively examined, thereby increasing the possibility of cost reduction. © 2018 John Wiley & Sons, Ltd.