Results 11 - 20
of
73
SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems
"... Many programs can be configured through dynamic and/or static selection of configuration variables. A software product line (SPL), for example, specifies a family of programs where each program is defined by a unique combination of features. Systematically testing SPL programs is expensive as it can ..."
Abstract
-
Cited by 9 (1 self)
- Add to MetaCart
(Show Context)
Many programs can be configured through dynamic and/or static selection of configuration variables. A software product line (SPL), for example, specifies a family of programs where each program is defined by a unique combination of features. Systematically testing SPL programs is expensive as it can require running each test against a combinatorial number of configurations. Fortunately, a test is often independent of many configuration variables and need not be run against every combination. Configurations that are not required for a test can be pruned from execution. This paper presents SPLat, a new way to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. SPLat achieves an optimal reduction in the number of configurations and is lightweight compared to prior work that used static analysis and heavyweight dynamic execution. Experimental results on 10 SPLs written in Java show that SPLat substantially reduces the total test execution time in many cases. Moreover, we demonstrate the scalability of SPLat by applying it to a large industrial code base written in Ruby on Rails.
Integration testing in software product line engineering: A model-based technique
- In Proceedings of the 10th Conference on Fundamental Approaches to Software Engineering (FASE 2007
, 2007
"... Abstract. The development process in software product line engineering is di- ..."
Abstract
-
Cited by 9 (1 self)
- Add to MetaCart
(Show Context)
Abstract. The development process in software product line engineering is di-
Exploiting Constraint Solving History to Construct Interaction Test Suites
, 2007
"... Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are const ..."
Abstract
-
Cited by 7 (0 self)
- Add to MetaCart
Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. In recent work, automated constraint solving methods have been combined with search-based CIT methods to address this problem with promising results. In this paper, we observe that the pattern of computation in greedy CIT algorithms leads to sequences of constraint solving problems that are closely related to one another. We propose two techniques for exploiting the history of constraint solving: (1) using incremental algorithms that are present within available constraint solvers and (2) mining constraint solver data structures to extract information that can be used to reduce the CIT search space. We evaluate the cost-effectiveness of these reductions on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those four systems. In combination our techniques reduce the cost of CIT in the presence of constraints to that of traditional unconstrained CIT methods without sacrificing the quality of solutions.
Specification-based testing for software product lines
- In SEFM 2008
, 2008
"... In this paper, we develop a testing theory for specification-based software product line development. Starting with a framework for the evaluation of test cases with respect to formal specifications, we develop a notion of enhancement, which allows to re-use test cases in a horizontal systems develo ..."
Abstract
-
Cited by 6 (3 self)
- Add to MetaCart
(Show Context)
In this paper, we develop a testing theory for specification-based software product line development. Starting with a framework for the evaluation of test cases with respect to formal specifications, we develop a notion of enhancement, which allows to re-use test cases in a horizontal systems development process. In such a process, more and more features are added to an existing software product. For specification-based testing, this means that the corresponding specifications are enhanced more and more, and that new test cases must be added to an existing test suite in order to test the additional features. We formally define an enhancement relation between CSP-CASL specifications, describe a test evaluation method for software product lines based on CSP-CASL specifications, and prove several preservation results which allow to re-use test cases. We illustrate our approach with the example of a product line of remote control units for consumer products. 1.
Automated Requirements-based Generation of Test Cases for Product Families
- in « Proc. of the 18th IEEE International Conference on Automated Software Engineering (ASE’03
, 2003
"... Software product families (PF) are becoming one of the key challenges of software engineering. Despite recent interest in this area, the extent to which the close relationship between PF and requirements engineering is exploited to guide the V&V tasks is still limited. In particular, PF processe ..."
Abstract
-
Cited by 6 (1 self)
- Add to MetaCart
(Show Context)
Software product families (PF) are becoming one of the key challenges of software engineering. Despite recent interest in this area, the extent to which the close relationship between PF and requirements engineering is exploited to guide the V&V tasks is still limited. In particular, PF processes generally lack support for generating test cases from requirements. In this paper, we propose a requirements-based approach to functional testing of product lines, based on a formal test generation tool. Here, we outline how product-specific test cases can be automatically generated from PF functional requirements expressed in UML. We study the efficiency of the generated test cases on a case study.
A survey on software product line testing
- In Proceedings of the 16th International Software Product Line ConferenceVolume 1 (2012), ACM
"... Software product line (SPL) testing consists of two separate but closely related test engineering activities: domain testing and application testing. Various software product line testing approaches have been developed over the last decade, and surveys have been conducted on them. However, thus far ..."
Abstract
-
Cited by 6 (0 self)
- Add to MetaCart
(Show Context)
Software product line (SPL) testing consists of two separate but closely related test engineering activities: domain testing and application testing. Various software product line testing approaches have been developed over the last decade, and surveys have been conducted on them. However, thus far none of them deeply addressed the questions of what researches have been conducted in order to overcome the challenges posed by the two separate testing activities and their relationships. Thus, this paper surveys the current software product line testing approaches by defining a reference SPL testing processes and identifying, based on them, key research perspectives that are important in SPL testing. Through this survey, we identify the researches that addressed the challenges and also derive open research opportunities from each perspective.
A Reuse Technique for Performance Testing of Software Product Lines
"... Testing that the applications of a software product line comply with their functional as well as with their nonfunctional requirements (for example performance) is important for achieving the desired product quality. Existing approaches for software product line testing only deal with testing an app ..."
Abstract
-
Cited by 5 (1 self)
- Add to MetaCart
(Show Context)
Testing that the applications of a software product line comply with their functional as well as with their nonfunctional requirements (for example performance) is important for achieving the desired product quality. Existing approaches for software product line testing only deal with testing an application against its functional requirements. In this paper we present a technique that supports the development of reusable performance test case scenarios in domain engineering and the reuse of these scenarios in application engineering. The technique is an extension of the ScenTED technique for system testing from our previous work. The technique focuses on load testing and performance profiling, two types of performance testing, and it has been validated in a case study at Siemens Medical Solutions HS IM. 1.
Regression Test Selection and Product Line SystemTesting
- In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST
"... Abstract — Context: Software product lines (SPL) are used in industry to achieve more efficient software development. To test a SPL is complex and costly and often becomes a bottleneck in the product line organization. Objective: This research aims to develop and evaluate strategies for improving sy ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
Abstract — Context: Software product lines (SPL) are used in industry to achieve more efficient software development. To test a SPL is complex and costly and often becomes a bottleneck in the product line organization. Objective: This research aims to develop and evaluate strategies for improving system test selection in a SPL. Method: Initially industrial practices and research in both SPL testing and traditional regression test selection have been surveyed. Two systematic literature reviews, two industrial exploratory surveys and one industrial evaluation of a pragmatic test selection approach have been conducted. Results: There is a lack of industrial evaluations as well as of useful solutions, both regarding regression test selection and SPL testing. Test selection is an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Conclusions: Continued research will be done in close cooperation with industry with the goal to define a tool for visualizing system test coverage in a product line and the delta between a product and the covered part of the product line. Keywords- software product line testing; regression testing; industrial practices, literature review, test case selection; test coverage I.
Testing Software Assets of Framework-Based Product Families during Application Engineering Stage
- Journal of Software
, 2008
"... Abstract — An application framework provides reusable design and implementation for a family of software systems. At the application engineering stage, application developers extend framework assets to build their particular framework instantiations. Typically, framework software assets are tested b ..."
Abstract
-
Cited by 4 (1 self)
- Add to MetaCart
(Show Context)
Abstract — An application framework provides reusable design and implementation for a family of software systems. At the application engineering stage, application developers extend framework assets to build their particular framework instantiations. Typically, framework software assets are tested before being used. However, achieving complete coverage of a system under test is impossible or at least impractical. Therefore, framework software assets can have undiscovered errors that appear in some instantiations. During the application engineering stage, it is important to identify the framework use cases that are used in the instantiation but not covered during the framework testing stage. In this paper, a testing model that considers retesting framework assets during the application engineering stage is proposed. In addition, a test-case-reusing technique is introduced to identify uncovered framework use cases and cover them by reusing the test cases already built during the framework domain engineering stage. Empirical studies are reported to show the adequacy of the proposed framework test-case-reusing technique in terms of reducing testing time and effort, and a supporting tool that automates the proposed test-case-reusing technique is developed and introduced. Index Terms — hooks, object-oriented framework, domain
White-Box Approaches for Improved Testing and Analysis of Configurable Software Systems
"... There is a significant conceptual gap between the source code of a configurable system and the runtime behaviors of its individual configurations. In the source, configurations are woven together into a conceptually unified program. At runtime, however, they are largely treated as independent execut ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
There is a significant conceptual gap between the source code of a configurable system and the runtime behaviors of its individual configurations. In the source, configurations are woven together into a conceptually unified program. At runtime, however, they are largely treated as independent executables. This gap leads to static analyses that, by acting on the source representing the entire configurable system, yield imprecise results with respect to individual executables. Testing, in contrast, acts on individual executables without leveraging the configurable codebase per se. In this paper, we sketch a research path that seeks to narrow the configuration source-runtime gap, based on the observation that most configurations share significant amounts of source-level structure (hence “white-box”) with other, related, configurations. We seek to identify and exploit this structure to reduce analysis and testing effort by sharing analysis and test results among related configurations. 1.