• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

Testing a Software Product Line (2001)

by J D McGregor
Add To MetaCart

Tools

Sorted by:
Results 11 - 20 of 73
Next 10 →

SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems

by Chang Hwan, Peter Kim, Darko Marinov, Sarfraz Khurshid, Don Batory
"... Many programs can be configured through dynamic and/or static selection of configuration variables. A software product line (SPL), for example, specifies a family of programs where each program is defined by a unique combination of features. Systematically testing SPL programs is expensive as it can ..."
Abstract - Cited by 9 (1 self) - Add to MetaCart
Many programs can be configured through dynamic and/or static selection of configuration variables. A software product line (SPL), for example, specifies a family of programs where each program is defined by a unique combination of features. Systematically testing SPL programs is expensive as it can require running each test against a combinatorial number of configurations. Fortunately, a test is often independent of many configuration variables and need not be run against every combination. Configurations that are not required for a test can be pruned from execution. This paper presents SPLat, a new way to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. SPLat achieves an optimal reduction in the number of configurations and is lightweight compared to prior work that used static analysis and heavyweight dynamic execution. Experimental results on 10 SPLs written in Java show that SPLat substantially reduces the total test execution time in many cases. Moreover, we demonstrate the scalability of SPLat by applying it to a large industrial code base written in Ruby on Rails.
(Show Context)

Citation Context

...age of configurations, but running all the configurations for all the tests can be prohibitively expensive. We leave it as a future work to explore a good strategy to sample from these configurations =-=[8, 9, 30]-=-. RQ3. Moreover, based on the fact that we could run SPLat on the codebase as large as Groupon PWA, we conclude the following: SPLat scales to real, large industrial code. The implementation effort fo...

Integration testing in software product line engineering: A model-based technique

by Sacha Reis, Andreas Metzger, Klaus Pohl - In Proceedings of the 10th Conference on Fundamental Approaches to Software Engineering (FASE 2007 , 2007
"... Abstract. The development process in software product line engineering is di- ..."
Abstract - Cited by 9 (1 self) - Add to MetaCart
Abstract. The development process in software product line engineering is di-
(Show Context)

Citation Context

...plications would have to be individually tested in domainsengineering, resulting in an enormous test effort.sIn the literature, several approaches for SPL testing have been proposed (e.g.,s[4][10][15]=-=[17]-=-[20][24]). However, these approaches cover unit and system testingsonly.sThis paper presents a model-based, automated technique for integration testing insdomain engineering. Our technique generates i...

Exploiting Constraint Solving History to Construct Interaction Test Suites

by Myra B. Cohen, et al. , 2007
"... Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are const ..."
Abstract - Cited by 7 (0 self) - Add to MetaCart
Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. In recent work, automated constraint solving methods have been combined with search-based CIT methods to address this problem with promising results. In this paper, we observe that the pattern of computation in greedy CIT algorithms leads to sequences of constraint solving problems that are closely related to one another. We propose two techniques for exploiting the history of constraint solving: (1) using incremental algorithms that are present within available constraint solvers and (2) mining constraint solver data structures to extract information that can be used to reduce the CIT search space. We evaluate the cost-effectiveness of these reductions on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those four systems. In combination our techniques reduce the cost of CIT in the presence of constraints to that of traditional unconstrained CIT methods without sacrificing the quality of solutions.

Specification-based testing for software product lines

by Temesghen Kahsai, Markus Roggenbach, Bernd-holger Schlingloff - In SEFM 2008 , 2008
"... In this paper, we develop a testing theory for specification-based software product line development. Starting with a framework for the evaluation of test cases with respect to formal specifications, we develop a notion of enhancement, which allows to re-use test cases in a horizontal systems develo ..."
Abstract - Cited by 6 (3 self) - Add to MetaCart
In this paper, we develop a testing theory for specification-based software product line development. Starting with a framework for the evaluation of test cases with respect to formal specifications, we develop a notion of enhancement, which allows to re-use test cases in a horizontal systems development process. In such a process, more and more features are added to an existing software product. For specification-based testing, this means that the corresponding specifications are enhanced more and more, and that new test cases must be added to an existing test suite in order to test the additional features. We formally define an enhancement relation between CSP-CASL specifications, describe a test evaluation method for software product lines based on CSP-CASL specifications, and prove several preservation results which allow to re-use test cases. We illustrate our approach with the example of a product line of remote control units for consumer products. 1.
(Show Context)

Citation Context

... late 1990’s (see, e.g., [11, 14]), and extensively studied subsequently in [10, 20], with annual conferences and a huge body of engineering literature in [4, 2]. Testing for SPLs was investigated in =-=[21, 17]-=- and others; the main focus of these papers is the informal or formal derivation of test cases from requirement and feature models. Formal methods for SPLs have been studied in [16] and others; wherea...

Automated Requirements-based Generation of Test Cases for Product Families

by Clémentine Nebut, Simon Pickin, Yves Le Traon, Jean-marc Jézéquel - in « Proc. of the 18th IEEE International Conference on Automated Software Engineering (ASE’03 , 2003
"... Software product families (PF) are becoming one of the key challenges of software engineering. Despite recent interest in this area, the extent to which the close relationship between PF and requirements engineering is exploited to guide the V&V tasks is still limited. In particular, PF processe ..."
Abstract - Cited by 6 (1 self) - Add to MetaCart
Software product families (PF) are becoming one of the key challenges of software engineering. Despite recent interest in this area, the extent to which the close relationship between PF and requirements engineering is exploited to guide the V&V tasks is still limited. In particular, PF processes generally lack support for generating test cases from requirements. In this paper, we propose a requirements-based approach to functional testing of product lines, based on a formal test generation tool. Here, we outline how product-specific test cases can be automatically generated from PF functional requirements expressed in UML. We study the efficiency of the generated test cases on a case study.
(Show Context)

Citation Context

...us task, which is in serious need of automation. Among the numerous test case generation techniques that can be found in the literature, few of them have been adapted to the PF context. The survey of =-=[5]-=- gives a detailed state of the art of PF testing and shows that until now, most test case generation has been done manually, the only test automation being in the generation of test scripts 1 This wor...

A survey on software product line testing

by Jihyun Lee, Sungwon Kang, Danhyung Lee - In Proceedings of the 16th International Software Product Line ConferenceVolume 1 (2012), ACM
"... Software product line (SPL) testing consists of two separate but closely related test engineering activities: domain testing and application testing. Various software product line testing approaches have been developed over the last decade, and surveys have been conducted on them. However, thus far ..."
Abstract - Cited by 6 (0 self) - Add to MetaCart
Software product line (SPL) testing consists of two separate but closely related test engineering activities: domain testing and application testing. Various software product line testing approaches have been developed over the last decade, and surveys have been conducted on them. However, thus far none of them deeply addressed the questions of what researches have been conducted in order to overcome the challenges posed by the two separate testing activities and their relationships. Thus, this paper surveys the current software product line testing approaches by defining a reference SPL testing processes and identifying, based on them, key research perspectives that are important in SPL testing. Through this survey, we identify the researches that addressed the challenges and also derive open research opportunities from each perspective.
(Show Context)

Citation Context

...hat use domain artifacts (O1-1)shave been proposed in [18, 20, 23, 26]. Combinatorial approachessbased on feature models, which are major domain artifacts ofsSPLE, are relevant to this classification =-=[8]-=-. The pair-wisescombination method, which greatly reduces the number ofscombinations necessary for testing, has been presented in [17, 18,s26, 39, 40]. However, this method also has a disadvantage in ...

A Reuse Technique for Performance Testing of Software Product Lines

by Sacha Reis, Andreas Metzger, Klaus Pohl
"... Testing that the applications of a software product line comply with their functional as well as with their nonfunctional requirements (for example performance) is important for achieving the desired product quality. Existing approaches for software product line testing only deal with testing an app ..."
Abstract - Cited by 5 (1 self) - Add to MetaCart
Testing that the applications of a software product line comply with their functional as well as with their nonfunctional requirements (for example performance) is important for achieving the desired product quality. Existing approaches for software product line testing only deal with testing an application against its functional requirements. In this paper we present a technique that supports the development of reusable performance test case scenarios in domain engineering and the reuse of these scenarios in application engineering. The technique is an extension of the ScenTED technique for system testing from our previous work. The technique focuses on load testing and performance profiling, two types of performance testing, and it has been validated in a case study at Siemens Medical Solutions HS IM. 1.
(Show Context)

Citation Context

... software product line engineering. Several approaches for the development of reusable test artifacts in domain engineering and the reuse of these artifacts in application engineering exist (e.g., [2]=-=[6]-=-[7][10]). However, these approaches focus on testing functional requirements only. There are currently no product line specific approaches that deal with testing performance or other non-functional re...

Regression Test Selection and Product Line SystemTesting

by Emelie Engström - In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST
"... Abstract — Context: Software product lines (SPL) are used in industry to achieve more efficient software development. To test a SPL is complex and costly and often becomes a bottleneck in the product line organization. Objective: This research aims to develop and evaluate strategies for improving sy ..."
Abstract - Cited by 4 (0 self) - Add to MetaCart
Abstract — Context: Software product lines (SPL) are used in industry to achieve more efficient software development. To test a SPL is complex and costly and often becomes a bottleneck in the product line organization. Objective: This research aims to develop and evaluate strategies for improving system test selection in a SPL. Method: Initially industrial practices and research in both SPL testing and traditional regression test selection have been surveyed. Two systematic literature reviews, two industrial exploratory surveys and one industrial evaluation of a pragmatic test selection approach have been conducted. Results: There is a lack of industrial evaluations as well as of useful solutions, both regarding regression test selection and SPL testing. Test selection is an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Conclusions: Continued research will be done in close cooperation with industry with the goal to define a tool for visualizing system test coverage in a product line and the delta between a product and the covered part of the product line. Keywords- software product line testing; regression testing; industrial practices, literature review, test case selection; test coverage I.
(Show Context)

Citation Context

...factors in the testing context affect thesefficiency of a test strategy?sIII. RELATED WORKsSoftware product line testing is a relatively new researchsarea. The first papers were published in 2001[10]]=-=[11]-=-, andsmost papers have been published in workshops andsconferences. There is a well established understanding aboutschallenges [9]. However, when looking for solutions to theseschallenges, we mostly f...

Testing Software Assets of Framework-Based Product Families during Application Engineering Stage

by Jehad Al Dallal, Paul Sorenson - Journal of Software , 2008
"... Abstract — An application framework provides reusable design and implementation for a family of software systems. At the application engineering stage, application developers extend framework assets to build their particular framework instantiations. Typically, framework software assets are tested b ..."
Abstract - Cited by 4 (1 self) - Add to MetaCart
Abstract — An application framework provides reusable design and implementation for a family of software systems. At the application engineering stage, application developers extend framework assets to build their particular framework instantiations. Typically, framework software assets are tested before being used. However, achieving complete coverage of a system under test is impossible or at least impractical. Therefore, framework software assets can have undiscovered errors that appear in some instantiations. During the application engineering stage, it is important to identify the framework use cases that are used in the instantiation but not covered during the framework testing stage. In this paper, a testing model that considers retesting framework assets during the application engineering stage is proposed. In addition, a test-case-reusing technique is introduced to identify uncovered framework use cases and cover them by reusing the test cases already built during the framework domain engineering stage. Empirical studies are reported to show the adequacy of the proposed framework test-case-reusing technique in terms of reducing testing time and effort, and a supporting tool that automates the proposed test-case-reusing technique is developed and introduced. Index Terms — hooks, object-oriented framework, domain
(Show Context)

Citation Context

...d technique to two framework instantiationsswere conducted. The experiments show the adequacy ofsthe proposed framework test-case-reusing technique insterms of reducing testing time and effort. As in =-=[8]-=-, thestechnique introduced in this paper assumes that thesframework test assets are provided with the framework. Assupporting tool called JFramework Re-Tester issdeveloped to automate the introduced t...

White-Box Approaches for Improved Testing and Analysis of Configurable Software Systems

by Marius Nita, David Notkin
"... There is a significant conceptual gap between the source code of a configurable system and the runtime behaviors of its individual configurations. In the source, configurations are woven together into a conceptually unified program. At runtime, however, they are largely treated as independent execut ..."
Abstract - Cited by 4 (0 self) - Add to MetaCart
There is a significant conceptual gap between the source code of a configurable system and the runtime behaviors of its individual configurations. In the source, configurations are woven together into a conceptually unified program. At runtime, however, they are largely treated as independent executables. This gap leads to static analyses that, by acting on the source representing the entire configurable system, yield imprecise results with respect to individual executables. Testing, in contrast, acts on individual executables without leveraging the configurable codebase per se. In this paper, we sketch a research path that seeks to narrow the configuration source-runtime gap, based on the observation that most configurations share significant amounts of source-level structure (hence “white-box”) with other, related, configurations. We seek to identify and exploit this structure to reduce analysis and testing effort by sharing analysis and test results among related configurations. 1.
(Show Context)

Citation Context

... configurability related to a focused market segment or mission. Testing and analysis of software product lines is not addressed aggressively in the literature; most focuses on processes for doing so =-=[8, 1]-=- rather than on testing or analysis per se. These approaches, each in its own way, define mechanisms and design approaches to reduce the complexity of a software system intended to represent a set of ...

Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University