Abstract-Regression testing is a necessary but expensive maintenance activity aimed at showing that code has not been adversely affected by changes. Regression test selection techniques reuse tests from an existing test suite to test a modified program. Many regression test selection techniques have been proposed; however, it is difficult to compare and evaluate these techniques because they have different goals. This paper outlines the issues relevant to regression test selection techniques, and uses these issues as the basis for a framework within which to evaluate the techniques. We illustrate the application of our framework by using it to evaluate existing regression test selection techniques. The evaluation reveals the strengths and weaknesses of existing techniques, and highlights some problems that future work in this area should address. Index Terms-Software maintenance, regression testing, selective retest, regression test selection. 1

Regression testing is an expensive but necessary maintenance activity performed on modified software to provide confidence that changes are correct and do not adversely affect other portions of the software. A regression test selection technique chooses, from an existing test set, tests that are deemed necessary to validate modified software. We present a new technique for regression test selection. Our algorithms construct control flow graphs for a procedure or program and its modified version, and use these graphs to select tests that execute changed code from the original test suite. We prove that under certain conditions, the set of tests our technique selects includes every test from the original test suite that can expose faults in the modified procedure or program. Under these conditions our algorithms are safe. Moreover, although our algorithms may select some tests that cannot expose faults, they are at least as precise as other safe regression test selection algorith...

Regression testing is the process of validating modified software to detect whether new errors have been introduced into previously tested code and to provide confidence that modifications are correct. Since regression testing is an expensive process, researchers have proposed regression test selection techniques as a way to reduce some of this expense. These techniques attempt to reduce costs by selecting and running only a subset of the test cases in a program's existing test suite. Although there have been some analytical and empirical evaluations of individual techniques, to our knowledge only one comparative study, focusing on one aspect of two of these techniques, has been reported in the literature. We conducted an experiment to examine the relative costs and benefits of several regression test selection techniques. The experiment examined ve techniques for reusing test cases, focusing on their relative abilities to reduce regression testing effort and uncover faults in modified programs. Our results highlight several differences between the techniques, and expose essential tradeoffs that should be considered when choosing a technique for practical application.

Software maintainers are faced with the task of regression testing: retesting a modified program on an often large number of test cases. The cost of regression testing can be reduced if the size of the program that must be retested is reduced and if old test cases and old test results can be reused. Tw o complimentary algorithms for reducing the cost of regression testing are presented. The first produces a program called differences that captures the semantic change between certified, apreviously tested program, and modified, achanged version of certified. Itis more efficient to test differences, because it omits unchanged computations. The program differences is computed using a combination of program slices. The second algorithm identifies test cases for which certified and modified will produce the same output and existing test cases that will test components new inmodified. Not rerunning test cases that produce the same output avoids unproductive testing; testing new components with existing test cases avoids the costly construction of new test cases. The second algorithm is based on the notion of common execution patterns, which is the interprocedural extension of the notion introduced by Bates and Horwitz. Program components with common execution patterns have the same execution pattern during some call to their procedure. They are computed using a new type of interprocedural slice called a calling context slice. Whereas an interprocedural slice includes the program components necessary to

The purpose of regression testing is to ensure that changes made to software, such as adding new features or modifying existing features, have not adversely affected features of the software that should not change. Regression testing is usually performed by running some, or all, of the test cases created to test modifications in previous versions of the software. Many techniques have been reported on how to select regression tests so that the number of test cases does not grow too large as the software evolves. Our proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions. This technique seeks to identify a representative subset of all test cases that may result in different output behavior on the new software version. We report our experience with a tool called ATAC which implements this technique. Keywords: Regression Testing, Modification-Based Test Selection, Test Set Minimization, Test Set Prioritization 1

This paper presents an algorithm that reduces the cost of regression testing by reducing the number of test cases that must be re-run and by reducing the size of the program that they must be run on. The algorithm uses dependence graphs and program slicing to partition the components of the new program into two sets: preserved points---components that have unchanged run-time behavior; and affected points---components that have changed run-time behavior. Only test cases that test the behavior of affected points must be re-run; the behavior of the preserved points is guaranteed to be the same in the old and new versions of the program. Furthermore, the algorithm produces a program differences, which captures the behavior of (only) the affected points. Thus, rather than re-testing the (large) new program on a large number of test cases, it is possible to certify the new program by running the (smaller) program differences on a (smaller) number of test cases. CR Categories and Subject De...

Regression testing is performed on modified software to provide confidence that changed and affected portions of the code behave correctly. We present an approach to regression testing that handles two important tasks: selecting tests from the existing test suite that should be rerun, and identifying portions of the code that must be covered by tests. Both tasks are performed by traversing graphs for the program and its modified version. We first apply our technique to single procedures and then show how our technique is applied at the interprocedural level. Our approach has several advantages over previous work. First, our test selection technique is safe, selecting every test that may produce different output in the modified program. However, our selection technique chooses smaller test sets than other safe approaches. Second, our approach is the first safe approach to identify coverage requirements, and the first safe approach to do so interprocedurally. Third, our approach handles ...

Regression testing is a necessary but costly maintenance activity aimed at demonstrating that code has not been adversely affected by changes. A selective approach to regression testing selects tests for a modified program from an existing test suite. We present a new technique for selective regression testing. Our algorithm constructs control dependence graphs for program versions, and uses these graphs to determine which tests from the existing test suite may exhibit changed behavior on the new version. Unlike most previous techniques for selective retest, our algorithm selects every test from the original test suite that might expose errors in the modified program, and does this without prior knowledge of program modifications. Our algorithm handles all language constructs and program modifications, and is easily automated. 1 Introduction It has been estimated that software maintenance activities account for as much as two-thirds of the overall cost of software production[18]. One...

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xiii 1. INTRODUCTION : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.1 Problem and Motivation : : : : : : : : : : : : : : : : : : : : : : : : 2 1.2 Scope of This Research : : : : : : : : : : : : : : : : : : : : : : : : : 2 1.3 Organization of This Dissertation : : : : : : : : : : : : : : : : : : : 5 2. AN OVERVIEW OF MUTATION AND DATA FLOW TESTING : : : : 9 2.1 Mutation Testing : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9 2.2 Alternate Mutation Testing : : : : : : : : : : : : : : : : : : : : : : 10 2.2.1 Randomly Selected x% Mutation Testing : : : : : : : : : : : 10 2.2.2 Constrained Mutation Testing : : : : : : : : : : : : : : : : : 11 2.2.3 Other Forms of Mutation Testing : : : : : : : : : : : : : : : 12 2.3 Data Flow Testing : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 2.4 Testing Tools : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16 3. A FORMALEVALUATIONOF M...

Regression testing is a commonly used activity whose purpose is to determine whether the modifications made to a software system have introduced new faults. For many large, complex, software systems the retest all strategy is not practical: the resources required to reexecute and verify all available test cases (i.e., time and human effort) are prohibitive. Ad hoc methods are not desirable, as they can compromise the reliability of the regression test activity and consequently the reliability of the software system being tested. In this paper we present a new technique for selecting regression test cases based on the modifications that have been made on the program. The technique, which is based on the idea of directly comparing source files from the old and the new version of the program, has been implemented in a tool called Pythia. A novel characteristic of Pythia, which is capable of analyzing large software systems written in C, is that it has been implemented primarily through th...