There are two main goals in testing software: (1) To achieve adequate quality (debug testing); the objective is to probe the software for defects so that these can be removed. (2) To assess existing quality (operational testing); the objective is to gain confidence that the software is reliable. The names are arbitrary, and most testing techniques address both goals to some degree. However, debug methods tend to ignore random selection of test data from an operational profile, while for operational methods this selection is all-important. Debug methods are thought, without any real proof, to be good at uncovering defects so that these can be repaired, but having done so they do not provide a technically defensible assessment of the reliability that results. On the other hand, operational methods provide accurate assessment, but may not be as useful for achieving reliability. This paper examines the relationship between the two testing goals, using a probabilistic analysis. We define s...

Testing is an important part of any software development project, and can typically surpass more than half of the development cost. For safety-critical computer based systems, testing is even more important due to stringent reliability and safety requirements. However, most safety-critical computer based systems are real-time systems, and the majority of current testing and debugging techniques have been developed for sequential (non real-time) programs. These techniques are not directly applicable to real-time systems, since they disregard issues of timing and concurrency. This means that existing techniques for reproducible testing and debugging cannot be used. Reproducibility is essential for regression testing and cyclic debugging, where the same test cases are run repeatedly with the intention of verifying modified program code or to track down errors. The current trend of consumer and industrial applications goes from single microcontrollers to sets of distributed micro-controllers, which are even more challenging than handling real-time per-see, since multiple loci of observation and control additionally must be considered. In this thesis we try to remedy these problems by presenting an integrated approach to monitoring, testing, and debugging of distributed real-time systems. For monitoring

Ideally, a measure of the security of a system should capture quantitatively the intuitive notion of ‘the ability of the system to resist attack’. That is, it should be operational, reflecting the degree to which the system can be expected to remain free of security breaches under particular conditions of operation (including attack). Instead, current security levels at best merely reflect the extensiveness of safeguards introduced during the design and development of a system. Whilst we might expect a system developed to a higher level than another to exhibit ‘more secure behaviour ’ in operation, this cannot be guaranteed; more particularly, we cannot infer what the actual security behaviour will be from knowledge of such a level. In the paper we discuss similarities between reliability and security with the intention of working towards measures of ‘operational security ’ similar to those that we have for reliability of systems. Very informally, these measures could involve expressions such as the rate of occurrence of security breaches (cf rate of occurrence of failures in reliability), or the probability that a specified ‘mission ’ can be accomplished without a security breach (cf reliability function). This new approach is based on the analogy between system failure and security breach. A number of other analogies to support this view are introduced. We examine this duality critically, and have identified a number of important open questions that need to be answered before this quantitative approach can be taken further. The work described here is therefore somewhat tentative, and one of our major intentions is to invite discussion about the plausibility and feasibility of this new approach.

participated in more than 30 industrial projects, published over 250 papers, and helped to develop many commercial systems and software tools. Professor Lyu is frequently invited as a keynote or tutorial speaker to conferences and workshops in U.S., Europe, and

Software reliability models are an important tool in quality management and release planning. There is a large number of di#erent models that often have strengths in di#erent areas. This paper proposes a model that is based on a geometric sequence of the failure rates of faults. This property of the failure process was observed in practice at Siemens among others and led to the development of the Fischer-Wagner model. It is described in detail and evaluated using standard criteria. Most importantly the model is able to perform equally well as other models in terms of its predictive validity.

In the development of software reliability measurement and prediction, many software reliability growth models have been proposed. Application of these models to real data sources has shown that there is commonly great disagreement in predictions, while none of them has been shown to be more trustworthy than others in terms of predictive quality in all applications. In this paper, we shall use various methods to get new predictions by combining the predictions obtained from different models. Two weight decision approaches which we call Bayesian Inference and Switching have been used in this combination. We shall show the resulting predictive quality by application of these techniques to a number of data sets. By using the combined prediction method we can get improved predictions or automatically choose the "best" prediction system from all the available prediction systems. The combination method has no specific requirements on the prediction systems being combined. Release...

OF THESIS ACCURATE SOFTWARE RELIABILITY ESTIMATION A large number of software reliability growth models are now available. It is widely known that none of these models performs well in all situations, and that choosing the appropriate model aprioriis difficult. For this reason recent work has focused on how these models can be made more accurate, rather than trying to find a model which works in all cases. This includes various efforts at data filtering and recalibration, and an examination of the physical interpretation of model parameters. Here we examine the impact of the parameter estimation technique on model accuracy, and show that the maximum likelihood method provides for estimates which are more reliable than the least squares method. We present an interpretation of the parameters for the popular logarithmic model, and show that it may be possible to use this interpretation to overcome some of the difficulties found in working with early failure test data. We present a new s...

In this paper, we propose a new paradigm for increasing the reliability of a software system by combining reactive and proactive approaches. The proposed approach employs rollback and restart for masking transient failures, and employs on-line software version change to remove faults from the software. A model for reliability analysis of a system employing the proposed approach is presented. The analysis shows that substantial benefit in reliability can be obtained by employing the proposed approach. A prototype system which incorporates the proposed approach is also described. 1. Introduction One way to increase the reliability of a software system is to deploy software fault tolerance. The earliest approaches for handling software faults employed design diversity. With such techniques, if a module cannot provide service, then other modules, which have different designs, are used to provide the required service. The two well known methods for organizing the different versions of a m...

Bayesian approach using nonhomogeneous Poisson process is considered for modeling software reliability problems. A generalized gamma and lognormal order statistics models are considered to model epochs of the failures of software. Metropolis algorithms along with Gibbs steps are proposed to perform the Bayesian inference of such models. Some Bayesian model diagnostics are developed and incorporated to verify further modeling assumptions. Model selection based on a prequential likelihood of conditional predictive ordinates is considered. The methodology developed in this paper is exemplified with a software reliability data set introduced by Jelinski and Moranda (1972). Key Words: Generalized Gamma intensity function, General order statistics, Gibbs sampling, Lognormal intensity function, Metropolis algorithm, Model diagnostics, Model selection, Nonhomogeneous Poisson process. AMS 1991 subject classifications: 62F15, 62N05. 1 Introduction The modelling of the number of failures of a...

Procedures for estimating the parameters of the general class of semiparametric models for recurrent events proposed by Peña and Hollander (2004) are developed. This class of models incorporates an effective age function which encodes the changes that occur after each event occurrence such as the impact of an intervention, it allows for the modeling of the impact of accumulating event occurrences on the unit, it admits a link function in which the effect of possibly time-dependent covariates are incorporated, and it allows the incorporation of unobservable frailty components which induce dependencies among the inter-event times for each unit. The estimation procedures are semiparametric in that a baseline hazard function is nonparametrically specified. The sampling distribution properties of the estimators are examined through a simulation study, and the consequences of mis-specifying the model are analyzed. The results indicate that the flexibility of this general class of models provides a safeguard for analyzing recurrent event data, even data possibly arising from a frailty-less mechanism. The estimation procedures are applied to real data sets arising in the biomedical and public health settings, as well as from reliability and engineering situations. In particular, the procedures are applied to a data set pertaining to times to recurrence of bladder cancer and the results of the analysis are compared to those obtained using three methods of analyzing recurrent event data.