Results 1 
7 of
7
Optimizing Requirements Decisions With KEYS
, 2008
"... ... for external access to five of JPL’s realworld requirements models, anonymized to conceal proprietary information, but retaining their computational nature. Experimentation with these models, reported herein, demonstrates a dramatic speedup in the computations performed on them. These models ha ..."
Abstract

Cited by 17 (8 self)
 Add to MetaCart
... for external access to five of JPL’s realworld requirements models, anonymized to conceal proprietary information, but retaining their computational nature. Experimentation with these models, reported herein, demonstrates a dramatic speedup in the computations performed on them. These models have a well defined goal: select mitigations that retire risks which, in turn, increases the number of attainable requirements. Such a nonlinear optimization is a wellstudied problem. However identification of not only (a) the optimal solution(s) but also (b) the key factors leading to them is less well studied. Our technique, called KEYS, shows a rapid way of simultaneously identifying the solutions and their key factors. KEYS improves on prior work by several orders of magnitude. Prior experiments with simulated annealing or treatment learning took tens of minutes to hours to terminate. KEYS runs much faster than that; e.g for one model, KEYS ran 13,000 times faster than treatment learning (40 minutes versus 0.18 seconds). Processing these JPL models is a nonlinear optimization problem: the fewest mitigations must be selected while achieving the most requirements. Nonlinear optimization is a well studied problem. With this paper, we challenge other members of the PROMISE community to improve on our results with other techniques.
Sawtooth: Learning from huge amounts of data
, 2004
"... Data scarcity has been a problem in data mining up until recent times. Now, in the era of the Internet and the tremendous advances in both, data storage devices and highspeed computing, databases are filling up at rates never imagined before. The machine learning problems of the past have been augm ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Data scarcity has been a problem in data mining up until recent times. Now, in the era of the Internet and the tremendous advances in both, data storage devices and highspeed computing, databases are filling up at rates never imagined before. The machine learning problems of the past have been augmented by an increasingly important one, scalability. Extracting useful information from arbitrarily large data collections or data streams is now of special interest within the data mining community. In this research we find that mining from such large datasets may actually be quite simple. We address the scalability issues of previous widelyused batch learning algorithms and discretization techniques used to handle continuous values within the data. Then, we describe an incremental algorithm that addresses the scalability problem of Bayesian classifiers, and propose a Bayesiancompatible online discretization technique that handles continuous values, both with a “simplicity first ” approach and very low memory (RAM) requirements. To my family. To Nana. iii iv
Finding Robust Solutions in Requirements Models
"... Abstract. Solutions to nonlinear requirements engineering problems may be “brittle”; i.e. small changes may dramatically alter solution effectiveness. Hence, it is not enough to just generate solutions to requirements problems we must also assess solution robustness. The KEYS2 algorithm can genera ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
(Show Context)
Abstract. Solutions to nonlinear requirements engineering problems may be “brittle”; i.e. small changes may dramatically alter solution effectiveness. Hence, it is not enough to just generate solutions to requirements problems we must also assess solution robustness. The KEYS2 algorithm can generate decision ordering diagrams. Once generated, these diagrams can assess solution robustness in linear time. In experiments with realworld requirements engineering models, we show that KEYS2 can generate decision ordering diagrams in O(N 2). When assessed in terms of terms of (a) reducing inference times, (b) increasing solution quality, and (c) decreasing the variance of the generated solution, KEYS2 outperforms other search algorithms (simulated annealing, ASTAR, MaxWalkSat). 1
Learning IV&V Strategies
"... Modern business practices are complex. Consider, for example, NASA's software IV&V (independent verification and validation) team that monitors a diverse range of complex software written by a wide range of contractors from around the world. In an effort to better understand the core busine ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
Modern business practices are complex. Consider, for example, NASA's software IV&V (independent verification and validation) team that monitors a diverse range of complex software written by a wide range of contractors from around the world. In an effort to better understand the core business of IV&V, the authors recently conducted DELPHI sessions with experienced IV&V analysts to build a model reflecting their understanding of what level of IV&V is appropriate for different projects. The resulting model, while short, contains subtle interactions that are not immediately apparent. To understand those interactions, we conducted Monte Carlo studies to grow data sets from the model. These data sets where summarized using TAR3 (a minimal contrast set learner) to discover (a) the core business decisions that decide what level of IV&V is appropriate; and (b) whether or not specializations of the problem domain can lead to more simple and robust models. 1.
A Baseline Method For SearchBased Software Engineering
"... Background: Searchbased Software Engineering (SBSE) uses a variety of techniques such as evolutionary algorithms or metaheuristic searches but lacks a standard baseline method. Aims: The KEYS2 algorithm meets the criteria of a baseline. It is fast, stable, easy to understand, and presents results ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Background: Searchbased Software Engineering (SBSE) uses a variety of techniques such as evolutionary algorithms or metaheuristic searches but lacks a standard baseline method. Aims: The KEYS2 algorithm meets the criteria of a baseline. It is fast, stable, easy to understand, and presents results that are competitive with standard techniques. Method: KEYS2 operates on the theory that a small subset of variables control the majority of the search space. It uses a greedy search and a Bayesian ranking heuristic to fix the values of these variables, which rapidly forces the search towards stable highscoring areas. Results: KEYS2 is faster than standard techniques, presents competitive results (assessed with a ranksum test), and offers stable solutions. Conclusions: KEYS2 is a valid candidate to serve as a baseline technique for SBSE research.
ModelBased Software Testing via Incremental Treatment Learning
"... Modelbased software has become quite popular in recent years, making its way into a broad range of areas, including the aerospace industry. The models provide an easy graphical interface to develop systems, which can generate the sometimes tedious code that follows. While there are many tools avail ..."
Abstract
 Add to MetaCart
(Show Context)
Modelbased software has become quite popular in recent years, making its way into a broad range of areas, including the aerospace industry. The models provide an easy graphical interface to develop systems, which can generate the sometimes tedious code that follows. While there are many tools available to assess standard procedural code, there are limits to the testing of modelbased systems. A major problem with the models are that their internals often contain gray areas of unknown system behavior. These possible behaviors form what is known as a data cloud, which is an overwhelming range of possibilities of a system that can overload analysts [3]. With large data clouds, it is hard to demonstrate which particular decision leads to a particular outcome. Even if definite decisions can’t be made, it is possible to reduce the variance of and condense the clouds [3]. This paper presents two case studies; one with a simple illustrative model and another with a more complex application. The TAR3 treatment learning tool summarizes the particular attribute ranges that selects for particular behaviors of interest, reducing the data clouds. 1
Minimal ContrastSet Learning, ModelBased Software Engineering, MetaHeuristic Search Abstract The Robust Optimization of NonLinear Requirements Models
, 2010
"... Solutions to nonlinear requirements engineering problems may be “brittle”; i.e. small changes may dramatically alter solution effectiveness. Hence, it is not enough to just generate solutions to requirements problems we must also assess solution robustness. This thesis aims to address two concerns ..."
Abstract
 Add to MetaCart
(Show Context)
Solutions to nonlinear requirements engineering problems may be “brittle”; i.e. small changes may dramatically alter solution effectiveness. Hence, it is not enough to just generate solutions to requirements problems we must also assess solution robustness. This thesis aims to address two concerns: (a) Is demonstrating robustness a time consuming task? and (b) Is it necessary that solution quality be traded off against solution robustness? Using a Bayesian ranking heuristic, the KEYS2 algorithm fixes a small number of important variables, rapidly pushing the search into a stable, optimal plateau. By design, KEYS2 generates decision ordering diagrams (in time experimentally shown to be O(N2)). Once generated, these diagrams can confirm solution robustness in linear time. When assessed in terms of reducing inference times, increasing solution quality, and decreasing the variance of the generated solution, KEYS2 outperforms other search algorithms (simulated annealing, A*, MaxWalkSat).