Citation Context

...argues that binary representation decomposes the chromosome into the largest number of smallest possible building blocks in order for the recombination and mutation operators to work most effectively =-=[15]-=-. However, this is disputed by Antonisse [16], who advocates the use of more expressive alphabets. Davis [17] supports this view. For nine real-world applications using Genetic Algorithms over a varie...

Citation Context

...hange in energy of the system when subjected to a cooling process, until it converges into a steady state. This algorithm was later proposed as the basis of the search mechanism by Kirkpatrick et al. =-=[6]-=-. 2.3 Evolutionary Algorithms Evolutionary Algorithms use simulated evolution as a search strategy to evolve candidate solutions, using operators inspired by genetics and natural selection. Genetic Al...

Citation Context

...ea of the search space. Conversely, too weak a strategy will result in too much exploration, and not enough evolution for the search to make substantial progress. Holland’s original Genetic Algorithm =-=[10]-=- used fitness-proportionate selection. In this selection mechanism, the expected number of times an individual is selected for reproduction is proportionate to the individual’s fitness in comparison w...

Citation Context

...rial is heated past its melting point, and then cooled back into a solid state, the structural properties of the cooled solid depend on the rate of cooling. An algorithm proposed by Metropolis et al. =-=[5]-=- simulates the change in energy of the system when subjected to a cooling process, until it converges into a steady state. This algorithm was later proposed as the basis of the search mechanism by Kir...

Citation Context

... by a small amount. In this way, an element of local search is incorporated [17]. Genetic Algorithms have been successfully applied to a wide range of problems. For introductory texts, see references =-=[15, 18]-=-. For shorter overviews and tutorials, see references [19, 9, 20]. 3 Structural (White-Box) Testing Structural, or white-box testing is the process of deriving tests from the internal structure of the...

Citation Context

... statically computed for each structural target. The variable dependence analysis information can also be used to compute a slice of the program with respect to the structural target. A program slice =-=[55]-=- is a smaller version of the original program which only contains the statements of interest according to some slicing criterion. In this case the criterion involves the removal of all statements that...

Citation Context

...ding blocks in order for the recombination and mutation operators to work most effectively [15]. However, this is disputed by Antonisse [16], who advocates the use of more expressive alphabets. Davis =-=[17]-=- supports this view. For nine real-world applications using Genetic Algorithms over a variety of problem domains, Davis found that real-valued representations always outperformed binary encodings (rea...

Citation Context

...stem, as described by some form of specification. 4.1 Generating Test Data from a Z Specification Jones et al. [59] generate test data for the triangle classification program, using a Z specification =-=[60]-=-. The state space of the system is described in a schema named T riangle0, which declares three input integer variables to represent the three sides of the triangle (x?, y? and z?). This schema also d...

Citation Context

...rated ideas from both traditions - narrowing the differences between the two. The discussion here, however, focuses on Genetic Algorithms. For more information on Evolution Strategies, see references =-=[7, 8, 9]-=-. 2.3.1 Genetic Algorithms The name “Genetic Algorithm” comes from the analogy between the encoding of candidate solutions as a sequence of simple components, and the genetic structure of a chromosome...

Citation Context

.... However, no path from node 13 is definition clear with respect to type. The term control dependency is used to describe the reliance of a node’s execution on the outcome at previous branching nodes =-=[21]-=-. A node z is postdominated by a node y in G if and only if every path from y to the exit node e contains z. Node z post-dominates a branch (y, x) if and only if every path from y to the exit node e t...

Citation Context

...tic structural test data generation is based on analysis of the internal structure of the program, without requiring that the program is actually executed. 3.2.1 Symbolic Execution Symbolic Execution =-=[22, 23]-=- is not the execution of a program in its true sense, but rather the process of assigning expressions to program variables as a path is followed through the code structure. The technique can be used t...

Citation Context

... of 1.0 to the median individual, and 2 − Z to the bottom individual. With a constant bias applied throughout the search, selective pressure is more constant and controlled [11]. Tournament Selection =-=[12]-=- is a noisy but fast rank selection algorithm. The population does not need to be sorted into fitness order. Two individuals are 6schosen at random from the population. A random number, 0 < r ≤ 1, is ...

Citation Context

... the top individual, a bias of 1.0 to the median individual, and 2 − Z to the bottom individual. With a constant bias applied throughout the search, selective pressure is more constant and controlled =-=[11]-=-. Tournament Selection [12] is a noisy but fast rank selection algorithm. The population does not need to be sorted into fitness order. Two individuals are 6schosen at random from the population. A ra...

Citation Context

...ues that have been used in software test data generation, namely Hill Climbing, Simulated Annealing and Evolutionary Algorithms. Further treatment of these search techniques can be found in reference =-=[3]-=-. The last decade has seen the emergence of many new techniques, which have not been exploited by the test data generation techniques presented here. Reference [4] gives treatment to some of these. 2....

Citation Context

... incorporated [17]. Genetic Algorithms have been successfully applied to a wide range of problems. For introductory texts, see references [15, 18]. For shorter overviews and tutorials, see references =-=[19, 9, 20]-=-. 3 Structural (White-Box) Testing Structural, or white-box testing is the process of deriving tests from the internal structure of the software under test. This section summarises some of the achieve...

Citation Context

...mplexity of program code. 3.2.2 Domain Reduction Domain reduction is a test data generation technique that was originally employed as part of Constraint-based Testing, developed by DeMillo and Offutt =-=[28]-=-. Constraint-based Testing builds up constraint systems which describe the given test goal. The solution to this constraint system brings about satisfaction of the goal. The original purpose of Constr...

Citation Context

...because the array index used in the assignment statement will be out of bounds: if (i >= 0 && i < size) { a[i] = 0; } It was not until 1990 that the ideas of Miller and Spooner were extended by Korel =-=[31]-=- for Pascal programs. In this work, the test data generation procedure worked on an instrumented version of the original program without the need for a straight-line version to be produced. The search...

Citation Context

...rated ideas from both traditions - narrowing the differences between the two. The discussion here, however, focuses on Genetic Algorithms. For more information on Evolution Strategies, see references =-=[7, 8, 9]-=-. 2.3.1 Genetic Algorithms The name “Genetic Algorithm” comes from the analogy between the encoding of candidate solutions as a sequence of simple components, and the genetic structure of a chromosome...

Citation Context

...rol dependence graph for the triangle classification program from Figure 4 constraint system in terms of the input variables which describes the conditions necessary for the traversal of a given path =-=[24, 25, 26]-=-. A forward traversal (or forward substitution) of a path, can be demonstrated with the triangle classification program in Figure 4. Say the path < s, 1, 5, 9, 10, 11, 12, 13, 14, e > is to be execute...

Citation Context

... incorporated [17]. Genetic Algorithms have been successfully applied to a wide range of problems. For introductory texts, see references [15, 18]. For shorter overviews and tutorials, see references =-=[19, 9, 20]-=-. 3 Structural (White-Box) Testing Structural, or white-box testing is the process of deriving tests from the internal structure of the software under test. This section summarises some of the achieve...

Citation Context

...s simply formed from the branch distance of the required branch. However, no guidance is provided so that the branch is actually reached within the program structure in the first place. McGraw et al. =-=[45]-=- alleviate this problem for condition coverage, by delaying an attempt to satisfy a condition within a branching expression until previous individuals have been already found which reach the branching...

Citation Context

... of Jones et al. [44] to loop testing falls into this category. Here, the objective function is simply the difference between the actual and desired number of iterations. In the work of Pargas et al. =-=[46]-=-, for statement and branch coverage, the control dependence graph of the test object is used. The sequence of control dependent nodes is identified for each structure. These are the branching nodes th...

Citation Context

...ded if the data dependencies of the test goal were also taken into account by search, and attempted to address this issue with the Chaining Approach. 3.3.4 The Chaining Approach The Chaining Approach =-=[34, 35]-=- uses the concept of an event sequence as an intermediate means of deciding the type of path required for execution up to the target node. An event sequence is basically a succession of program nodes ...

Citation Context

...ce values. However, this scheme suffers from similar problems suffered by the work of Miller and Spooner. Furthermore, the need to select a path is a burden on the tester. In the work of Jones et al. =-=[44]-=- for obtaining branch coverage, a path does not need to be selected. The objective function is simply formed from the branch distance of the required branch. However, no guidance is provided so that t...

Citation Context

...g space. For example, in Standard Binary Reflected Gray Code, 7 is represented as 0100, and 8 as 1100. Empirical evidence has shown that Gray codes are generally superior to standard binary encodings =-=[13, 14]-=-. Goldberg argues that binary representation decomposes the chromosome into the largest number of smallest possible building blocks in order for the recombination and mutation operators to work most e...

Citation Context

...rol dependence graph for the triangle classification program from Figure 4 constraint system in terms of the input variables which describes the conditions necessary for the traversal of a given path =-=[24, 25, 26]-=-. A forward traversal (or forward substitution) of a path, can be demonstrated with the triangle classification program in Figure 4. Say the path < s, 1, 5, 9, 10, 11, 12, 13, 14, e > is to be execute...

Citation Context

...Using this information, the search is directed into potentially promising areas of the search space. Search-based software test data generation is just one example of searchbased software engineering =-=[1, 2]-=-. To date, metaheuristic search techniques have been applied to automate test data generation in the following areas: • the coverage of specific program structures, as part of a structural, or white-b...

Citation Context

...rol dependence graph for the triangle classification program from Figure 4 constraint system in terms of the input variables which describes the conditions necessary for the traversal of a given path =-=[24, 25, 26]-=-. A forward traversal (or forward substitution) of a path, can be demonstrated with the triangle classification program in Figure 4. Say the path < s, 1, 5, 9, 10, 11, 12, 13, 14, e > is to be execute...

Citation Context

... limited to the use of relational operators only. 4.2 Testing Specification Conformance The last section showed how test data could be generated from a formal specification. The work of Tracey et al. =-=[61, 47]-=- extends this idea. In their technique the conformance of the implementation to its specification is checked by executing the test object with the generated test data, and then validating the output a...

Citation Context

...riented approach. However, search times increase, and the local search employed can still become trapped in difficult search spaces. 3.4 Applying Simulated Annealing The work of Tracey and co-authors =-=[36, 37]-=- applies Simulated Annealing to structural test data generation, in the hope of overcoming some of the problems associated with the application of local search. In this work, test data can be generate...

Citation Context

...the chromosome into the largest number of smallest possible building blocks in order for the recombination and mutation operators to work most effectively [15]. However, this is disputed by Antonisse =-=[16]-=-, who advocates the use of more expressive alphabets. Davis [17] supports this view. For nine real-world applications using Genetic Algorithms over a variety of problem domains, Davis found that real-...

Citation Context

...f all three variables being selected with the same value is 1 in 10,000. In such cases a more directed search technique is required to locate test data. 3.3.2 Applying Local Search Miller and Spooner =-=[30]-=- were the first to combine the results of actual executions of the program with a search technique. Their method was originally designed for the generation of floating-point test data, however the pri...

Citation Context

...Using this information, the search is directed into potentially promising areas of the search space. Search-based software test data generation is just one example of searchbased software engineering =-=[1, 2]-=-. To date, metaheuristic search techniques have been applied to automate test data generation in the following areas: • the coverage of specific program structures, as part of a structural, or white-b...

Citation Context

...ve guidance regarding the fact that the true branch from node 2 needs to be taken for the target statement to be reached. This results in poor search performance. The approach taken by Baresel et al. =-=[49]-=- is to treat branches that miss the target in iterations of the loop as if they were critical branches (recall that these branches are classed as semi-critical in Korel’s work). Thus, node 3 is treate...

Citation Context

...rated ideas from both traditions - narrowing the differences between the two. The discussion here, however, focuses on Genetic Algorithms. For more information on Evolution Strategies, see references =-=[7, 8, 9]-=-. 2.3.1 Genetic Algorithms The name “Genetic Algorithm” comes from the analogy between the encoding of candidate solutions as a sequence of simple components, and the genetic structure of a chromosome...

Citation Context

...T. If a test case is found that violates the timing constraints, the search can be terminated. Wegener et al. [66] were the first to apply Genetic Algorithms to temporal testing. In their experiments =-=[67, 68]-=- it is shown that Genetic Algorithms yield better results than random testing. A number of experiments with industrial test objects were carried out. A further experiment investigated six time-critica...

Citation Context

...T. If a test case is found that violates the timing constraints, the search can be terminated. Wegener et al. [66] were the first to apply Genetic Algorithms to temporal testing. In their experiments =-=[67, 68]-=- it is shown that Genetic Algorithms yield better results than random testing. A number of experiments with industrial test objects were carried out. A further experiment investigated six time-critica...

Citation Context

...ake use of both branch distance and control infor-60 27 -40 -20 0 20 j 40 60sObjective Function Value 70 60 50 40 30 20 10 0 -60 -40 i -20 0 20 40 60 Figure 14: Objective function landscape of Tracey =-=[47]-=- for example of Figure 11 Objective Function Value 2 1.5 1 0.5 0 -60 -40 i -20 0 20 40 60 Figure 15: Objective function landscape of Wegener et al. [48] for example of Figure 11 -60 -60 28 -40 -40 -20...

Citation Context

...re derived using Symbolic Execution, the method suffers from similar problems involving loops, procedure calls and computed storage locations. Dynamic Domain Reduction was introduced by Offutt et al. =-=[29]-=- with the intent of addressing some of these issues. Although called Dynamic Domain Reduction, the technique still has the characteristic that the program is not executed with real input values. As wi...

Citation Context

...aths, and the maximum number of iterations for each loop appearing in the code. Unfortunately, the possibility of simulation errors and the need for human involvement make this an error-prone process =-=[64, 65]-=-. The result produced can also be extremely pessimistic in the case of WCET and optimistic in the case of BCET. Sometimes the estimates can vary from those observed in practice by a magnitude of ten t...

Citation Context

...state structure of the system. 5 Grey-Box Testing Grey-box testing combines both structural and functional information for the purposes of testing. 5.1 Assertion Testing The work of Korel and Al-Yami =-=[62]-=- attempts to find test cases that violate assertion conditions, which can be embedded by the programmer into the program code. Assertions specify constraints that apply to some state of a computation....

Citation Context

...cript for the execution of the target structure. A further problem with state-based systems is their tendency to make use of flag and enumeration variables to control the current state. McMinn et al. =-=[57]-=- suggest an approach combining the evolutionary search for test data with the construction of event sequences in a similar style to the Chaining Approach. The construction of an event sequence can be ...

Citation Context

...re then produced. With these experiments, it was found that inputs could be found to violate an assertion - and thereby reveal a fault - 92% of the time. 5.2 Exception Condition Testing Tracey et al. =-=[63, 47]-=- built on the ideas of Korel and Al-Yami, using Genetic Algorithms and Simulated Annealing to generate input data to test the handling of run-time error conditions in code. In many languages, such as ...

Citation Context

...ul sub-path. For example at node 5, it is more attractive to manipulate c rather than a or b, since changing a or b may change the current successful sub-path through node 1. Gallagher and Narasimhan =-=[32]-=- built on Korel’s work for programs written in Ada. In particular, this was the first work to record support for the use of logical connectives within branch predicates. For predicates of the form A a...

Citation Context

...g space. For example, in Standard Binary Reflected Gray Code, 7 is represented as 0100, and 8 as 1100. Empirical evidence has shown that Gray codes are generally superior to standard binary encodings =-=[13, 14]-=-. Goldberg argues that binary representation decomposes the chromosome into the largest number of smallest possible building blocks in order for the recombination and mutation operators to work most e...

Citation Context

...ferred to in the literature as Evolutionary Testing (for example References [38, 39, 40]). The first work applying Evolutionary Algorithms to generate structural test data is that of Xanthakis et al. =-=[41]-=-. Up until this point, work on structural test data generation had largely focused on finding input data for specific paths or individual structures with programs, such as branches or statements. Init...

Citation Context

...tic structural test data generation is based on analysis of the internal structure of the program, without requiring that the program is actually executed. 3.2.1 Symbolic Execution Symbolic Execution =-=[22, 23]-=- is not the execution of a program in its true sense, but rather the process of assigning expressions to program variables as a path is followed through the code structure. The technique can be used t...

Citation Context

...riented approach. However, search times increase, and the local search employed can still become trapped in difficult search spaces. 3.4 Applying Simulated Annealing The work of Tracey and co-authors =-=[36, 37]-=- applies Simulated Annealing to structural test data generation, in the hope of overcoming some of the problems associated with the application of local search. In this work, test data can be generate...

Citation Context

...minimum value of the individual objective values of the expressions. 17s3.3.3 The Goal-Oriented Approach In his paper published in 1992, Korel developed what became known as the GoalOriented Approach =-=[33]-=-. All of the techniques concentrate on the execution of a path. For fulfilling a structural coverage criterion like statement coverage, this means a path has to be selected for each individual uncover...

Citation Context

...gorithms 3.5 Applying Evolutionary Algorithms The application of Evolutionary Algorithms to test data generation is often referred to in the literature as Evolutionary Testing (for example References =-=[38, 39, 40]-=-). The first work applying Evolutionary Algorithms to generate structural test data is that of Xanthakis et al. [41]. Up until this point, work on structural test data generation had largely focused o...

Citation Context

...can be discarded once the required test data has been found. A disadvantage of the approach is that it can not yet transform programs where flags are involved in loops. The approach of Baresel et al. =-=[51]-=- is to identify a sequence of nodes to be executed prior to the branch predicate containing the flag. For the example of Figure 22 where the true branch from node 4 is required, is it clear that node ...

Citation Context

...to maximise the objective function in the case of WCET, and minimise it in the case of BCET. If a test case is found that violates the timing constraints, the search can be terminated. Wegener et al. =-=[66]-=- were the first to apply Genetic Algorithms to temporal testing. In their experiments [67, 68] it is shown that Genetic Algorithms yield better results than random testing. A number of experiments wit...

Citation Context

...cation of metaheuristic search techniques to the testing of the logical behaviour of a system, as described by some form of specification. 4.1 Generating Test Data from a Z Specification Jones et al. =-=[59]-=- generate test data for the triangle classification program, using a Z specification [60]. The state space of the system is described in a schema named T riangle0, which declares three input integer v...

Citation Context

...ikely to be covered by chance, for example deeply nested structures, or branch predicates that are only true when an input variable has to be a specific value from a large domain. The work of Watkins =-=[43]-=- attempts to obtain full path coverage for programs. The objective function penalises individuals that follow already covered paths, by assigning a value that is the inverse of the number of times the...

Citation Context

...gorithms 3.5 Applying Evolutionary Algorithms The application of Evolutionary Algorithms to test data generation is often referred to in the literature as Evolutionary Testing (for example References =-=[38, 39, 40]-=-). The first work applying Evolutionary Algorithms to generate structural test data is that of Xanthakis et al. [41]. Up until this point, work on structural test data generation had largely focused o...

Citation Context

...t the end of the function call until the next time it is executed. Therefore, branching node 2 requires at least five executions of the function for the true branch to become feasible. Baresel et al. =-=[56]-=- aim to circumvent this for branch coverage by encoding individuals as sequences of input vectors. The sequence is of length n, in order for n calls to the function to be performed. Since the function...

Citation Context

... of the conditions at once. In this situation, care needs to be taken when side effects appear in any of the conditions. A solution here might be to apply a side-effect removal program transformation =-=[52, 53]-=- first. Alternatively, variables values could be saved into temporary variables inserted immediately before the branching statement, and restored after the side-effect if the condition would not norma...

Citation Context

...gorithms 3.5 Applying Evolutionary Algorithms The application of Evolutionary Algorithms to test data generation is often referred to in the literature as Evolutionary Testing (for example References =-=[38, 39, 40]-=-). The first work applying Evolutionary Algorithms to generate structural test data is that of Xanthakis et al. [41]. Up until this point, work on structural test data generation had largely focused o...

Citation Context

...be down to the use of system calls, linkage and compiler optimisation whose effects on temporal behaviour could only be guessed with difficulty by the developers. Additional work by O’Sullivan et al. =-=[70]-=- applies cluster analysis to determine when the search should be terminated. This technique decides if the search is converging on the basis of the distribution of individuals in the search space. Pus...

Citation Context

... of the conditions at once. In this situation, care needs to be taken when side effects appear in any of the conditions. A solution here might be to apply a side-effect removal program transformation =-=[52, 53]-=- first. Alternatively, variables values could be saved into temporary variables inserted immediately before the branching statement, and restored after the side-effect if the condition would not norma...

Citation Context

...erted immediately before the branching statement, and restored after the side-effect if the condition would not normally have been evaluated. 3.5.5 Applying Variable Dependence Analysis Harman et al. =-=[54]-=- apply variable dependence analysis to determine the subset of input variables that can not affect the outcome at a branch predicate. In this way, the search space can be reduced, increasing the chanc...

Citation Context

...aths, and the maximum number of iterations for each loop appearing in the code. Unfortunately, the possibility of simulation errors and the need for human involvement make this an error-prone process =-=[64, 65]-=-. The result produced can also be extremely pessimistic in the case of WCET and optimistic in the case of BCET. Sometimes the estimates can vary from those observed in practice by a magnitude of ten t...

Citation Context

...ration can be categorised on the basis of objective function construction (Figure 12). Coverage-Oriented Approaches reward individuals on the basis of covered program structures. In the work of Roper =-=[42]-=-, an individual is rewarded on the basis of the number of structures executed in accordance with the coverage criterion. Under this scheme, however, the search tends to reward individuals that execute...

Citation Context

...ed. 40sFurther possible areas of research include programs using information from files and sockets. Some initial work on structural testing of distributed systems includes that of Ferguson and Korel =-=[58]-=-. 4 Functional (Black-Box) Testing This section discusses the application of metaheuristic search techniques to the testing of the logical behaviour of a system, as described by some form of specifica...

Citation Context

... yield better results than random testing. A number of experiments with industrial test objects were carried out. A further experiment investigated six time-critical tasks in an engine control system =-=[69]-=-. Genetic Algorithms were again found to outperform random search, and also tests constructed by the developers themselves. The developer’s tests never found the longest execution times, and in three ...

Citation Context

... probability, the chance of a search technique executing it is low. If this branch is involved in a path leading to an extreme execution time, then the extreme execution time will not be found. Gross =-=[71]-=- identifies a number of properties of programs which lead to low probability branches, for example high levels of nesting, branches that are only executed if an input variable is a specific value, and...

Citation Context

...ch-based execution time testing has so far been largely restricted to programs of a procedural nature. However, some work extending these techniques to object-oriented software is beginning to appear =-=[73]-=-. 6.2 Future Directions for Search-based Non-Functional Testing Work in non-functional testing has been largely restricted to execution time testing. However, there are many more possibilities for aut...

Citation Context

...ded if the data dependencies of the test goal were also taken into account by search, and attempted to address this issue with the Chaining Approach. 3.3.4 The Chaining Approach The Chaining Approach =-=[34, 35]-=- uses the concept of an event sequence as an intermediate means of deciding the type of path required for execution up to the target node. An event sequence is basically a succession of program nodes ...

Citation Context

... that no cues from either the source code, machine code or details regarding the hardware to be used have been included for the benefit of the search. Gross also suggests the input of human knowledge =-=[72]-=-. Conversely, search-based techniques could be used to verify path feasibility for static analysis. Other strategies, such as guaranteeing survival of a path for a number of generations have been sugg...