In the past five years there has been a dramatic increase in work on Search Based Software Engineering (SBSE), an approach to software engineering in which search based optimisation algorithms are used to address problems in Software Engineering. SBSE has been applied to problems throughout the Software Engineering lifecycle, from requirements and project planning to maintenance and re-engineering. The approach is attractive because it offers a suite of adaptive automated and semi-automated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives. This paper 1 provides a review and classification of literature on SBSE. The paper identifies research trends and relationships between the techniques applied and the applications to which they have been applied and highlights gaps in the literature and avenues for further research.

A dependence cluster is a set of program statements, all of which are mutually inter-dependent. This paper reports a large scale empirical study of dependence clusters in C program source code. The study reveals that large dependence clusters are surprisingly commonplace. Most of the 45 programs studied have clusters of dependence that consume more than 10 % of the whole program. Some even have clusters consuming 80 % or more. The widespread existence of clusters has implications for source code analyses such as program comprehension, software maintenance, software testing, reverse engineering, reuse, and parallelization.

Existing automated test data generation techniques tend to start from scratch, implicitly assuming no pre-existing test data are available. However, this assumption may not always hold, and where it does not, there may be a missed opportunity; perhaps the pre-existing test cases could be used to assist the automated generation of additional test cases. This paper introduces search-based test data augmentation, a technique that can generate additional test data from existing test data using a meta-heuristic search algorithm. The proposed technique is compared to a widely studied test data generation approach in terms of both efficiency and effectiveness. The empirical evaluation shows that test data augmentation can be up to two orders of magnitude more efficient than existing test data generation techniques, while achieving comparable effectiveness in terms of structural coverage and mutation score. 1.

Dependence analysis underpins many activities in software maintenance such as comprehension and impact analysis. As a result, dependence has been studied widely for programming languages, notably through work on program slicing. However, there is comparatively little work on dependence analysis at the model level and hitherto, no empirical studies. We introduce a slicing tool for Extended Finite State Machines (EFSMs) and use the tool to gather empirical results on several forms of dependence found in ten EFSMs, including well-known benchmarks in addition to real-world EFSM models. We investigate the statistical properties of dependence using statistical tests for correlation and formalize and prove four of the empirical findings arising from our empirical study. The paper thus provides the maintainer with both empirical data and foundational theoretical results concerning dependence in EFSM models.

Despite the upsurge of interest in the Aspect-Oriented Programming (AOP) paradigm, there remain few results on test data generation techniques for AOP. Furthermore, there is no work on searchbased optimization for test data generation, an approach that has been shown to be successful in other programming paradigms. In this paper, we introduce a search-based optimization approach to automated test data generation for structural coverage of AOP systems. We present the results of an empirical study that demonstrates the effectiveness of the approach. We also introduce a domain reduction approach for AOP testing and show that this approach not only reduces test effort, but also increases test effectiveness. This finding is significant, because similar studies for non-AOP programming paradigms show no such improvement in effectiveness, merely a reduction in effort. We also present the results of an empirical study of the reduction in test effort achieved by focusing specifically on branches inside aspects.

Summary. Evolutionary Testing is an emerging methodology for automatically generating high quality test data. The focus of this paper is on presenting an approach for generating test cases for the unit-testing of object-oriented programs, with basis on the information provided by the structural analysis and interpretation of Java bytecode and on the dynamic execution of the instrumented test object. The rationale for working at the bytecode level is that even when the source code is unavailable, insight can still be obtained and used to guide the search-based test case generation process. Test cases are represented using the Strongly Typed Genetic Programming paradigm, which effectively mimics the polymorphic relationships, inheritance dependences and method argument constraints of object-oriented programs. 1

Abstract—Evolutionary white-box software testing has been extensively researched but is not yet applied in industry. In order to investigate the reasons for this, we evaluated a prototype version of a tool, representing the state-of-the-art for evolutionary structural testing, which is targeted at industrial use. The focus was on the applicability of the structural test tool in the industrial context and not on assessment of the test cases generated. Four case studies, each consisting of an embedded software module from the automotive industry implemented in the C language, were evaluated with the tool. The case studies had to be customized to cope with the limitations of the tool and in all, test case generation succeeded for 37 % of the functions selected for the evaluation. Weaknesses of the tool were reported to the developers and subsequently eliminated, resulting in a later version of the tool being able to process 82 % of the selected case study functions. However, the study shows that significant engineering work is still required before evolutionary structural testing is ready for industrial application. Keywords-structural testing; white-box testing; evolutionary testing; tool support; test automation. I.

Testability transformation (tetra) seeks to transform a program in order to make it easier to generate test data. The test data is generated from the transformed version of the program, but it is applied to the original version for testing purposes. A transformation is a testability transformation with respect to a test adequacy criterion if all test data that is adequate for the transformed program is also adequate for the untransformed program. Testability transformation has been shown to be effective at improving coverage for search–based test data generation. However, there are many interesting open problems. This paper presents some of these open problems. The aim is to show how testability transformation can be applied to a wide range of testing scenarios. 1

Ensuring security of software and computerized systems is a pervasive problem plaguing companies and institutions and affecting many areas of modern life. Software vulnerability may jeopardize information confidentiality and cause software failure leading to catastrophic threats to humans or severe economic losses. Size, complexity, extensibility, connectivity and the search for cheap systems make it very hard or even impossible to manually tackle vulnerability detection. Search based software testing attempts to solve two aspects of the cost- vulnerability problem. First, it’s cheaper because it is far less labor intensive when compared to traditional testing techniques. As a result, it can be used to more thoroughly test software and reduce the risk that a vulnerability slips into production code. Also, search based software testing can be specifically tailored to tackle the subset of well known security vulnerabilities responsible for most security threats. This paper is divided into two parts. It examines promising search based testing approaches to detecting software vulnerabilities, and then presents some of the most interesting open research problems.

Search-based test case generation for object-oriented software is hindered by the size of the search space, which encompasses the arguments to the implicit and explicit parameters of the test object’s public methods. The performance of this type of search problems can be enhanced by the definition of adequate Input Domain Reduction strategies. The focus of our on-going work is on employing evolutionary algorithms for generating test data for the structural unit-testing of Java programs. Test cases are represented and evolved using the Strongly-Typed Genetic Programming paradigm; Purity Analysis is particularly useful in this situation because it provides a means to automatically identify and remove Function Set entries that do not contribute to the definition of interesting test scenarios.