Mutation testing is a technique for unit testing software that, although powerful, is computationally expensive. The principal expense of mutation is that many variants of the test program, called mutants, must be repeatedly executed. This paper quantifies the expense of mutation in terms of the number of mutants that are created, then proposes and evaluates a technique that reduces the number of mutants by an order of magnitude. Selective mutation reduces the cost of mutation testing by reducing the number of mutants. This paper reports experimental results that compare selective mutation testing with standard, or non-selective, mutation testing, and results that quantify the savings achieved by selective mutation testing. The results support the hypothesis that selective mutation is almost as strong as non-selective mutation; in experimental trials selective mutation provides almost the same coverage as non-selective mutation, with a four-fold or more reduction in the number of mutants.

Fault-based testing strategies test software by focusing on specific, common types of faults. The coupling effect hypothesizes that test data sets that detect simple types of faults are sensitive enough to detect more complex types of faults. This paper describes empirical investigations into the coupling effect over a specific class of software faults. All of the results from this investigation support the validity of the coupling effect. The major conclusion from this investigation is the fact that by explicitly testing for simple faults, we are also implicitly testing for more complicated faults, giving us con dence that fault-based testing is an effective way to test software.

Abstract not found

Abstract not found

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xiii 1. INTRODUCTION : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.1 Problem and Motivation : : : : : : : : : : : : : : : : : : : : : : : : 2 1.2 Scope of This Research : : : : : : : : : : : : : : : : : : : : : : : : : 2 1.3 Organization of This Dissertation : : : : : : : : : : : : : : : : : : : 5 2. AN OVERVIEW OF MUTATION AND DATA FLOW TESTING : : : : 9 2.1 Mutation Testing : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9 2.2 Alternate Mutation Testing : : : : : : : : : : : : : : : : : : : : : : 10 2.2.1 Randomly Selected x% Mutation Testing : : : : : : : : : : : 10 2.2.2 Constrained Mutation Testing : : : : : : : : : : : : : : : : : 11 2.2.3 Other Forms of Mutation Testing : : : : : : : : : : : : : : : 12 2.3 Data Flow Testing : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 2.4 Testing Tools : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16 3. A FORMALEVALUATIONOF M...

Mutation testing is a technique for unit-testing software that, although powerful, is computationally expensive. The principal expense of mutation is that many variants of the test program, called mutants, must be repeatedly executed. Selective mutation is a way to approximate mutation testing that saves execution by reducing the number of mutants that must be executed. This paper reports experimental results that compare selective mutation testing to standard, or non-selective, mutation testing. The results support the hypothesis that selective mutation is almost as strong as non-selective mutation; in experimental trials selective mutation provides almost the same coverage as non-selective mutation, with significant reductions in cost. Fifteenth International Conference on Software Engineering, pages 100--107, Baltimore, Maryland, May 1993. 1 Introduction Mutation testing is a technique, originally proposed in 1978 [DLS78], that asks the tester to demonstrate that the test progr...

Security vulnerabilities often result from buffer overflows. A testing technique that instruments programs with code that keeps track of memory buffers, and checks arguments to functions to determine if they satisfy certain conditions, warns when a buffer overflow may occur. It does so when executed with ”normal ” test data as opposed to test data designed to trigger buffer overflows. A tool using this method was developed and evaluated by testing three widely used, open source software packages. This evaluation shows that the tool is useful for finding buffer overflow flaws, that it has a low false positive rate, and compares well with other techniques. 1

Mutation testing is a fault-based technique for unit level software testing. Weak mutation was proposed as a way to reduce the expense of mutation testing. Unfortunately, weak mutation is also expected to provide a weaker test of the software than mutation testing does. This paper presents results from an implementation of weak mutation, which we used to evaluate the effectiveness versus the efficiency of weak mutation. Additionally, we examined several options in an attempt to find the most appropriate way to implement weak mutation. Our results indicate that weak mutation can be applied in a manner that is almost as effective as mutation testing, and with significant computational savings.

Mutation testing is a technique for unit testing software that, although powerful, is computationally expensive. The principal expense of mutation is that many variants of the test program, called mutants, must be repeatedly executed. Selective mutation is a way to reduce the cost of mutation testing by reducing the number of mutants that must be executed. This paper reports experimental results that compare selective mutation testing to standard, or non-selective, mutation testing. The results support the hypothesis that selective mutation is almost as strong as non-selective mutation; in experimental trials selective mutation provides almost the same coverage as non-selective mutation, with a four-fold or more reduction in cost.

Abstract The Godzilla automatic test data generator is an integrated collection of tools that implements a relatively new test data generation method, constraint-based testing, that is based on mutation analysis. Constraint-based testing integrates mutation analysis with several other testing techniques, including statement coverage, branch coverage, domain perturbation and symbolic evaluation. Because Godzilla uses a rule-based approach to generate test data, it is easily extendible to allow new testing techniques to be integrated into the current system.