Experiments to Assess the Cost-Benefits of Test-Suite Reduction

Test-suite reduction techniques attempt to reduce the cost of saving and reusing test cases during software maintenance by eliminating redundant test cases from test suites. A potential drawback of these techniques is that in reducing a test suite they might reduce the ability of that test suite to reveal faults in the software. Previous studies suggested that test-suite reduction techniques can reduce test suite size without significantly reducing the fault-detection capabilities of test suites. To further investigate this issue we performed experiments in which we examined the costs and benefits of reducing test suites of various sizes for several programs and investigated factors that influence those costs and benefits. In contrast to the previous studies, our results reveal that the fault-detection capabilities of test suites can be severely compromised by test-suite reduction