Results 1  10
of
10
A Survey of Combinatorial Testing
, 2011
"... Combinatorial Testing (CT) can detect failures triggered by interactions of parameters in the Software Under Test (SUT) with a covering array test suite generated by some sampling mechanisms. It has been an active field of research in the last twenty years. This article aims to review previous work ..."
Abstract

Cited by 47 (1 self)
 Add to MetaCart
Combinatorial Testing (CT) can detect failures triggered by interactions of parameters in the Software Under Test (SUT) with a covering array test suite generated by some sampling mechanisms. It has been an active field of research in the last twenty years. This article aims to review previous work on CT, highlights the evolution of CT, and identifies important issues, methods, and applications of CT, with the goal of supporting and directing future practice and research in this area. First, we present the basic concepts and notations of CT. Second, we classify the research on CT into the following categories: modeling for CT, test suite generation, constraints, failure diagnosis, prioritization, metric, evaluation, testing procedure and the application of CT. For each of the categories, we survey the motivation, key issues, solutions, and the current state of research. Then, we review the contribution from different research groups, and present the growing trend of CT research. Finally, we recommend directions for future CT research, including: (1) modeling for CT, (2) improving the existing test suite generation algorithm, (3) improving analysis of testing result, (4) exploring the application of CT to different levels of testing and additional types of systems, (5) conducting more empirical studies to fully understand limitations and strengths of CT, and (6) combining CT with other testing techniques.
Combinatorial aspects of covering arrays
 LE MATEMATICHE (CATANIA
, 2004
"... Covering arrays generalize orthogonal arrays by requiring that ttuples be covered, but not requiring that the appearance of ttuples be balanced. Their uses in screening experiments has found application in software testing, hardware testing, and a variety of fields in which interactions among fact ..."
Abstract

Cited by 35 (9 self)
 Add to MetaCart
(Show Context)
Covering arrays generalize orthogonal arrays by requiring that ttuples be covered, but not requiring that the appearance of ttuples be balanced. Their uses in screening experiments has found application in software testing, hardware testing, and a variety of fields in which interactions among factors are to be identified. Here a combinatorial view of covering arrays is adopted, encompassing basic bounds, direct constructions, recursive constructions, algorithmic methods, and applications.
A Framework of Greedy Methods for Constructing Interaction Test Suites
 Intl. Conf. on Software Engineering (ICSE’05
, 2005
"... Greedy algorithms for the construction of software interaction test suites are studied. A framework is developed to evaluate a large class of greedy methods that build suites one test at a time. Within this framework are many instantiations of greedy methods generalizing those in the literature. Gre ..."
Abstract

Cited by 33 (8 self)
 Add to MetaCart
(Show Context)
Greedy algorithms for the construction of software interaction test suites are studied. A framework is developed to evaluate a large class of greedy methods that build suites one test at a time. Within this framework are many instantiations of greedy methods generalizing those in the literature. Greedy algorithms are popular when the time for test suite construction is of paramount concern. We focus on the size of the test suite produced by each instantiation. Experiments are analyzed using statistical techniques to determine the importance of the implementation decisions within the framework. This framework provides a platform for optimizing the accuracy and speed of "onetestatatime" greedy methods.
A densitybased greedy algorithm for higher strength covering arrays
, 2008
"... Algorithmic construction of software interaction test suites has focussed on pairwise coverage; less is known about the efficient construction of test suites for tway interactions with t ≥ 3. This work extends an efficient densitybased algorithm for pairwise coverage to generate tway interaction ..."
Abstract

Cited by 13 (4 self)
 Add to MetaCart
(Show Context)
Algorithmic construction of software interaction test suites has focussed on pairwise coverage; less is known about the efficient construction of test suites for tway interactions with t ≥ 3. This work extends an efficient densitybased algorithm for pairwise coverage to generate tway interaction test suites, and show that it guarantees a logarithmic upper bound on the size of the test suites as a function of the number of factors. To complement this theoretical guarantee, an implementation is outlined and some practical improvements are made. Computational comparisons with other published methods are reported. Many of the results improve upon those in the literature. However, limitations on the ability of onetestatatime algorithms are also identified.
Rouxtype constructions for covering arrays of strengths three and four
, 2006
"... A covering array CA(N; t; k; v) is an N k array such that every N t subarray contains all ttuples from v symbols at least once, where t is the strength of the array. Covering arrays are used to generate software test suites to cover all tsets of component interactions. Recursive constructions f ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
(Show Context)
A covering array CA(N; t; k; v) is an N k array such that every N t subarray contains all ttuples from v symbols at least once, where t is the strength of the array. Covering arrays are used to generate software test suites to cover all tsets of component interactions. Recursive constructions for covering arrays of strengths 3 and 4 are developed, generalizing many "Rouxtype" constructions. A numerical comparison with current construction techniques is given through existence tables for covering arrays.
Profiles of covering arrays of strength two
, 2013
"... Covering arrays of strength two have been widely studied as combinatorial models of software interaction test suites for pairwise testing. While numerous algorithmic techniques have been developed for the generation of covering arrays with few columns (factors), the construction of covering arrays ..."
Abstract
 Add to MetaCart
Covering arrays of strength two have been widely studied as combinatorial models of software interaction test suites for pairwise testing. While numerous algorithmic techniques have been developed for the generation of covering arrays with few columns (factors), the construction of covering arrays with many factors and few tests by these techniques is problematic. Random generation techniques can overcome these computational difficulties, but for strength two do not appear to yield a number of tests that is competitive with the fewest known. Consequently, effective construction of covering arrays with many factors and few tests relies on recursive construction techniques.
Construct Pairwise Test Suites Based on the BakSneppen Model of Biological Evolution
"... Abstract—Pairwise testing, which requires that every combination of valid values of each pair of system factors be covered by at lease one test case, plays an important role in software testing since many faults are caused by unexpected 2way interactions among system factors. Although metaheuristi ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Pairwise testing, which requires that every combination of valid values of each pair of system factors be covered by at lease one test case, plays an important role in software testing since many faults are caused by unexpected 2way interactions among system factors. Although metaheuristic strategies like simulated annealing can generally discover smaller pairwise test suite, they may cost more time to perform search, compared with greedy algorithms. We propose a new method, improved Extremal Optimization (EO) based on the BakSneppen (BS) model of biological evolution, for constructing pairwise test suites and define fitness function according to the requirement of improved EO. Experimental results show that improved EO gives similar size of resulting pairwise test suite and yields an 85 % reduction in solution time over SA.
Pairwise Test Suite Generator Tool Based On Harmony Search Algorithm (HSPTSGT)
"... plays an important role and required the most financial resources. However, exhaustive testing that required lot of time and financial resources is practically impossible. In this research we apply pairwise testing technique to reduce the number of test cases, by develop a pairwise test suite genera ..."
Abstract
 Add to MetaCart
(Show Context)
plays an important role and required the most financial resources. However, exhaustive testing that required lot of time and financial resources is practically impossible. In this research we apply pairwise testing technique to reduce the number of test cases, by develop a pairwise test suite generator tool based on Harmony Search Algorithm. Pairwise test cases were generates using harmony search algorithm that known with good balance and free from divergence. In our research, HSPTSGT has the ability to execute automatic mapping procedure to generate symbolic values from parameter values and convert it back in the final test suite. From the comparison against other pairwise testing tools, HSPTSGT can effectively generate optimum test suite with minimized test data and better than other tools on certain input specification.
Interaction Testing: From Pairwise to Variable Strength Interaction
"... Abstract—Although desirable as an important activity for quality assurances and enhancing reliability, complete and exhaustive software testing is prohibitively impossible due to resources as well as timing constraints. While earlier work has indicated that uniform pairwise testing (i.e. based on 2 ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Although desirable as an important activity for quality assurances and enhancing reliability, complete and exhaustive software testing is prohibitively impossible due to resources as well as timing constraints. While earlier work has indicated that uniform pairwise testing (i.e. based on 2way interaction of variables) can be effective to detect most faults in a typical software system, a counter argument suggests such conclusion cannot be generalized to all software system faults. In some system, faults may also be nonuniform and caused by more than two parameters. Considering these issues, this paper explores the issues pertaining to tway testing from pairwise to variable strength interaction in order to highlight the stateoftheart as well as the current stateofpractice. Keywordstway testing; interaction testing; variable strength interaction; I.