Abstract

This paper addresses the problem of reducing the size of test sets for regression testing and test output inspection. Since regression testing requires the execution of some, and in the worst case, all test cases, reducing the number of tests can have a large benefit. Additionally, testers generally have to examine the output of each test case, both during initial and regression testing. Since this is done by hand, reducing the number of outputs that need to be examined can reduce the cost of testing. We observe that for mutation-based test sets, the order in which the test cases are executed impacts the size of the test sets. This paper presents several strategies for selecting a smaller number of test cases by reordering the test tests. We illustrate our technique using a proof-of-concept empirically study using mutation testing, achieving approximately a 33 % reduction in size, and a corresponding reduction in the cost of regression testing, with a cost of only one extra run of the test case set. We suggest that these results should be extendable to apply to any test strategy that includes a quantifiable measure of test case effectiveness, such as data flow testing and branch testing, and try it with statement coverage with positive results.

Citations