Abstract-Regression testing is a necessary but expensive maintenance activity aimed at showing that code has not been adversely affected by changes. Regression test selection techniques reuse tests from an existing test suite to test a modified program. Many regression test selection techniques have been proposed; however, it is difficult to compare and evaluate these techniques because they have different goals. This paper outlines the issues relevant to regression test selection techniques, and uses these issues as the basis for a framework within which to evaluate the techniques. We illustrate the application of our framework by using it to evaluate existing regression test selection techniques. The evaluation reveals the strengths and weaknesses of existing techniques, and highlights some problems that future work in this area should address. Index Terms-Software maintenance, regression testing, selective retest, regression test selection. 1

Regression testing is an expensive but necessary maintenance activity performed on modified software to provide confidence that changes are correct and do not adversely affect other portions of the software. A regression test selection technique chooses, from an existing test set, tests that are deemed necessary to validate modified software. We present a new technique for regression test selection. Our algorithms construct control flow graphs for a procedure or program and its modified version, and use these graphs to select tests that execute changed code from the original test suite. We prove that under certain conditions, the set of tests our technique selects includes every test from the original test suite that can expose faults in the modified procedure or program. Under these conditions our algorithms are safe. Moreover, although our algorithms may select some tests that cannot expose faults, they are at least as precise as other safe regression test selection algorith...

Types of code changes in an object oriented library are described. A formal model for capturing and inferencing on the changes to identify affected classes is described. The model consists of three types of diagrams: the object relation diagram (ORD), the block branch diagram (BBD), and the object state diagram (OSD). An ORD describes the inheritance, aggregation, and association relationships between the classes of a C++ library. A BBD describes the control structure and interfaces of a member function. And an OSD describes the state behavior of a class. Unlike in modeling, these diagrams are automatically generated from code and facilitate understanding and changing a C++ library. An OO software maintenance environment that implements the research result is described. Our experience with the environment prototype shows promising results.

Regression testing is an expensive testing process used to re-validate software as it evolves. Various methodologies for improving regression testing processes have been explored, but the cost-effectiveness of these methodologies has been shown to vary with characteristics of regression test suites. One such characteristic involves the way in which test inputs are composed into test cases within a test suite. This article reports the results of controlled experiments examining the effects of two factors in test suite composition — test suite granularity and test input grouping — on the costs and benefits of several regression-testing-related methodologies: retest-all, regression test selection, test suite reduction, and test case prioritization. These experiments consider the application of several specific techniques, from each of these methodologies, across ten releases each of two substantial software systems, using seven levels of test suite granularity and two types of test input grouping. The effects of granularity, technique, and grouping on the cost and fault-detection effectiveness of regression testing under the given methodologies are analyzed. This analysis shows that test suite granularity significantly affects several cost-benefit factors for the methodologies considered, while test input grouping has limited effects. Further, the results expose essential tradeoffs affecting the relationship between test suite design and regression testing cost-effectiveness, with several implications for practice. 1

Many tools and techniques for addressing software maintenance problems rely on code coverage information. Often, this coverage information is gathered for a specific version of a software system, and then used to perform analyses on subsequent versions of that system without being recalculated. As a software system evolves, however, modifications to the software alter the software’s behavior on particular inputs, and code coverage information gathered on earlier versions of a program may not accurately reflect the coverage that would be obtained on later versions. This discrepancy may affect the success of analyses dependent on code coverage information. Despite the importance of coverage information in various analyses, in our search of the literature we find no studies specifically examining the impact of software evolution on code coverage information. Therefore, we conducted empirical studies to examine this impact. The results of our studies suggest that even relatively small modifications can greatly affect code coverage information, and that the degree of impact of change on coverage may be difficult to predict. 1

In the past five years there has been a dramatic increase in work on Search Based Software Engineering (SBSE), an approach to software engineering in which search based optimisation algorithms are used to address problems in Software Engineering. SBSE has been applied to problems throughout the Software Engineering lifecycle, from requirements and project planning to maintenance and re-engineering. The approach is attractive because it offers a suite of adaptive automated and semi-automated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives. This paper 1 provides a review and classification of literature on SBSE. The paper identifies research trends and relationships between the techniques applied and the applications to which they have been applied and highlights gaps in the literature and avenues for further research.

Test case prioritization techniques schedule test cases for regression testing in an order that increases their ability to meet some performance goal. One performance goal, rate of fault detection, measures how quickly faults are detected within the testing process. Previous work has provided a metric, AP F D, for measuring rate of fault detection, and techniques for prioritizing test cases in order to improve AP F D. This metric and these techniques, however, assume that all test case and fault costs are uniform. In practice, test case and fault costs can vary, and in such cases the previous AP F D metric and techniques designed to improve AP F D can be unsatisfactory. This paper presents a new metric for assessing the rate of fault detection of prioritized test cases, AP F DC, that incorporates varying test case and fault costs. The paper also describes adjustments to previous prioritization techniques that allow them, too, to be “cognizant ” of these varying costs. These techniques enable practitioners to perform a new type of prioritization: cost-cognizant test case prioritization. Finally, the results of a formative case study are presented. This study was designed to investigate the cost-cognizant metric and techniques and how they compare to their non-cost-cognizant counterparts. The study’s results provide insights regarding the use of cost-cognizant test case prioritization in a variety of real-world settings.