Results 1 - 10
of
30
Fault Prediction Using Early Lifecycle Data”,
- ISSRE 2007, the 18th IEEE Symposium on Software Reliability Engineering, IEEE Computer Society, Sweden,
, 2007
"... ..."
How Good is your Blind Spot Sampling Policy
- in Proc. of 8 th IEEE Int’l Symp. on High Assurance Systems Eng
, 2004
"... Assessing software costs money and better assessment costs exponentially more money. Given finite budgets, assessment resources are typically skewed towards areas that are believed to be mission critical. This leaves blind spots: portions of the system that may contain defects which may be missed. T ..."
Abstract
-
Cited by 14 (9 self)
- Add to MetaCart
(Show Context)
Assessing software costs money and better assessment costs exponentially more money. Given finite budgets, assessment resources are typically skewed towards areas that are believed to be mission critical. This leaves blind spots: portions of the system that may contain defects which may be missed. Therefore, in addition to rigorously assessing mission critical areas, a parallel activity should sample the blind spots. This paper assesses defect detectors based on static code measures as a blind spot sampling method. In contrast to previous results, we find that such defect detectors yield results that are stable across many applications. Further, these detectors are inexpensive to use and can be tuned to the specifics of the current business situations. 1
Mining repositories to assist in project planning and resource allocation
- Proceedings 1st International Workshop on Mining Software Repositories (MSR’04). University of Waterloo: Waterloo ON, 2004; 75–79. Available at: http://plg.uwaterloo.ca/∼aeehassa/home/pubs/MSR2004ProceedingsFINAL IEE Acrobat4.pdf [1 February 2007]. Copyri
"... Software repositories plus defect logs are useful for learning defect detectors. Such defect detectors could be a useful resource allocation tool for software managers. One way to view our detectors is that they are a V&V tool for V&V; i.e. they can be used to assess if ”too much ” of the te ..."
Abstract
-
Cited by 10 (3 self)
- Add to MetaCart
(Show Context)
Software repositories plus defect logs are useful for learning defect detectors. Such defect detectors could be a useful resource allocation tool for software managers. One way to view our detectors is that they are a V&V tool for V&V; i.e. they can be used to assess if ”too much ” of the testing budget is going to ”too little” of the system. Finding such detectors could become the business case that constructing a local repository is useful. Three counter arguments to such a proposal are (1) no general conclusions have been reported in any such repository despite years of effort; (2) if such general conclusions existed then there would be no need to build a local repository; (3) no such general conclusions will ever exist, according to many researchers. This article is a reply to these three arguments. Submitted to the International Workshop on Mining Software
Just enough learning (of association rules
- In WVU CSEE tech report, 2002. Available from http://tim.menzies.com/pdf/02tar2.pdf
"... An over-zealous machine learner can automatically generate large, intricate, theories which can be hard to understand. However, such intricate learning is not necessary in domains that lack complex relationships. A much simpler learner can suffice in domains with narrow funnels; i.e. where most doma ..."
Abstract
-
Cited by 8 (6 self)
- Add to MetaCart
(Show Context)
An over-zealous machine learner can automatically generate large, intricate, theories which can be hard to understand. However, such intricate learning is not necessary in domains that lack complex relationships. A much simpler learner can suffice in domains with narrow funnels; i.e. where most domain variables are controlled by a very small subset. Such a learner is TAR2: a weighted-class minimal contrast-set association rule learner that utilizes confidence-based pruning, but not support-based pruning. TAR2 learns treatments; i.e. constraints that can change an agent’s environment. Treatments take two forms. Controller treatments hold the smallest number of conjunctions that most improve the current state of the system. Monitor treatments hold the smallest number of conjunctions that best detect future faulty system behavior. Such treatments tell an agent what to do (apply the controller) and what to watch for (the monitor conditions) within the current environment. Because TAR2 generates very small theories, our experience has been that users prefer its tiny treatments. The success of such a simple learner suggests that many domains lack complex relationships.
First contract: Better, earlier decisions for software projects
- In Submitted to Proceedings of the Fifth International Symposium on Requirements Engineering, 2001. Available from http://tim.menzies.com/pdf/01first.pdf
"... Decisions made in the earliest phases of software development have the greatest effect on the likelihood of project success. These decisions are used to establish the requirements of the product under development, determine budget and schedule, plan the allocation of resources to the development, et ..."
Abstract
-
Cited by 6 (6 self)
- Add to MetaCart
(Show Context)
Decisions made in the earliest phases of software development have the greatest effect on the likelihood of project success. These decisions are used to establish the requirements of the product under development, determine budget and schedule, plan the allocation of resources to the development, etc. This decision making is very challenging, given the impediments of incomplete information, absence of a shared vision, hard-to-discern ramifications of choices, etc. This paper describes an ongoing effort to address these challenges by means of a synergistic collaboration of research tools, techniques and best-practice knowledge. The goals of this effort are to give the end users more soundly based estimates, enhanced abilities to make trades amongst requirements, risks, and development, and better optimized development plans. To achieve these goals, our effort combines research in the areas of estimation, planning, visualization, elicitation, negotiation, machine learning, abduction, and knowledge representation. A hypothetical scenario is used to introduce and illustrate our approach. We describe the key contributions that each of our research efforts provides, and go on to consider the synergistic benefits of their combination. Realization of this combination is well underway; status and future plans are described. Finally, this collaborative work is related to existing research and practice, to show how our approach furthers the ability to make critical decisions in the early phases of software development projects.
How to argue less
"... Requirements engineering can stall if all stakeholder dis-putes are explored; e.g. a mere 20 boolean options im-plies possible arguments. One method of reducing this argument space is to focus the arguments on core issues and ignore the peripheral arguments. The-oretical and experimental studies s ..."
Abstract
-
Cited by 5 (3 self)
- Add to MetaCart
(Show Context)
Requirements engineering can stall if all stakeholder dis-putes are explored; e.g. a mere 20 boolean options im-plies possible arguments. One method of reducing this argument space is to focus the arguments on core issues and ignore the peripheral arguments. The-oretical and experimental studies strongly suggest that, in the usual case, a space of arguments contains many irrel-evant and repeated disputes. Hence, the space of all criti-cal arguments may be dramatically smaller than the space of all arguments. It is argued here that this reduction can be implemented via abduction plus induction. Abduction can extract the consistent conclusions derivable from a re-quirements model. Induction can learn from that sample the attributes that most change the behaviour of the model. Experiments with this abduction-plus-induction approach have found that a very small number of critical factors can be found within seemingly huge argument spaces. A strong theoretical case can be made that this approach will apply to many domains and scale to very large models. 1.
Learning early lifecycle IV&V quality indicators
- In IEEE Metrics ’03, 2003. Available from http://menzies.us/pdf/03early.pdf
"... Traditional methods of generating quality code indicators (e.g. linear regression, decision tree induction) can be demonstrated to be inappropriate for IV&V purposes. IV&V is a unique aspect of the software lifecycle, and different methods are necessary to produce quick and accurate results. ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
Traditional methods of generating quality code indicators (e.g. linear regression, decision tree induction) can be demonstrated to be inappropriate for IV&V purposes. IV&V is a unique aspect of the software lifecycle, and different methods are necessary to produce quick and accurate results. If quality code indicators could be produced on a per-project basis, then IV&V could proceed in a more straight-forward fashion, saving time and money. This article presents one case study on just such a project, showing that by using the proper metrics and machine learning algorithms, quality indicators can be found as early as 3 months into the IV&V process. 1
On the value of learning from defect dense components for software defect prediction
- InProceedings of the 6th International Conference on Predictive Models in Software Engineering (p. 14). ACM
, 2010
"... BACKGROUND: Defect predictors learned from static code measures can isolate code modules with a higher than usual probability of defects. AIMS: To improve those learners by focusing on the defect-rich portions of the training sets. METHOD: Defect data CM1, KC1, MC1, PC1, PC3 was separated into compo ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
BACKGROUND: Defect predictors learned from static code measures can isolate code modules with a higher than usual probability of defects. AIMS: To improve those learners by focusing on the defect-rich portions of the training sets. METHOD: Defect data CM1, KC1, MC1, PC1, PC3 was separated into components. A subset of the projects (selected at random) were set aside for testing. Training sets were generated for a NaiveBayes classifier in two ways. In sample the dense treatment, the components with higher than the median number of defective modules were used for training. In the standard treatment, modules from any component were used for training. Both samples were run against the test set and evaluated using recall, probability of false alarm, and precision. In addition, under sampling and over sampling was performed on the defect data. Each method was repeated in a 10-by-10 cross-validation experiment. RESULTS: Prediction models learned from defect dense components out-performed standard method, under sampling, as well as over sampling. In statistical rankings based on recall, probability of false alarm, and precision, models learned from dense components won 4-5 times more often than any other method, and also lost the least amount of times. CONCLUSIONS: Given training data where most of the defects exist in small numbers of components, better defect predictors can be trained from the defect dense components.
An Alternative to Model Checking: Verification by Random Search of AND-OR Graphs Representing Finite-State Models
- In Proc. International Symposium on High-Assurance Systems Engineering
, 2002
"... In the development of high-assurance systems, formal modeling, analysis and verification techniques are play-ing an increasingly important role. In spite of signifi-cant advances, formal modeling and verification still suffers from limited applicability due to exponential runtime space growth exibit ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
In the development of high-assurance systems, formal modeling, analysis and verification techniques are play-ing an increasingly important role. In spite of signifi-cant advances, formal modeling and verification still suffers from limited applicability due to exponential runtime space growth exibited by model checkers. In this paper, we describe an alternative to model check-ing. We describe an algorithm that automatically trans-lates Finite State Machine models used by model checkers into AND-OR graphs. State space verification of AND-OR graphs does not suffer from state space explosion, but its exhaustive search is an NP complete problem. Hence, we demonstrate that random searches of AND-OR graphs are a vaible alternative to model checking, suitable for system debugging and fast analysis during system development. We support our conclusions through the studies of two models, Dekker’s two process mutual exclusion algorithm and the Space Shuttle’s liquid hydrogen subsystem. 1
Finding faults quickly in formal models using random search
- In Proc. of SEKE
, 2003
"... As software grows more complex, automatic verifi-cation tools become increasingly important. Unfortu-nately many systems are large enough that complete verification requires a lot of time and memory, if it is possible at all. In our preliminary studies, random search, although not a complete techniq ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
As software grows more complex, automatic verifi-cation tools become increasingly important. Unfortu-nately many systems are large enough that complete verification requires a lot of time and memory, if it is possible at all. In our preliminary studies, random search, although not a complete technique, was able to find most faults significantly faster and with less mem-ory than would be required for full verification. Here we present an experiment in which random search was used to find faults in fault-seeded models of a large com-mercial flight guidance system. To assess the perfor-mance of random search we compared it to a full ver-ification done by the model checker NuSMV. The ran-dom search results were surprisingly complete, finding nearly 90 % of the faults reported by NuSMV—and these results were generated faster and using less memory. We suggest that random search be used in conjunction with verification tools, perhaps as a fast debugging tool during model development, or even as an alternative model checking strategy on models for which the time and memory requirements would make full verification impossible. 1
