Empirical studies of test-suite reduction

Abstract

Test-suite reduction techniques attempt to reduce the costs of saving and reusing test cases during software maintenance by eliminating redundant test cases from test suites. A potential drawback of these techniques is that reducing the size of a test suite might reduce its ability to reveal faults in the software. Previous studies have suggested that test-suite reduction techniques can reduce test-suite size without significantly reducing the fault-detection capabilities of test suites. These studies, however, involved particular programs and types of test suites, and to begin to generalize their results, further work is needed. This paper reports on the design and execution of additional studies, examining the costs and benefits of test-suite reduction, and the factors that influence these costs and benefits. In contrast to previous studies, results of these studies reveal that the fault-detection capabilities of test suites can be severely compromised by test-suite reduction.