Modern society depends on computers for a number of critical tasks in which failure can have very high costs. As a consequence, high levels of dependability (reliability, safety, etc.) are required from such computers, including their software. Whenever a quantitative approach to risk is adopted, these requirements must be stated in quantitative terms, and a rigorous demonstration of their being attained is necessary. For software used in the most critical roles, such demonstrations are not usually supplied. The fact is that the dependability requirements often lie near the limit of the current state of the art, or beyond, in terms not only of the ability to satisfy them, but also, and more often, of the ability to demonstrate that they are satisfied in the individual operational products (validation). We discuss reasons why such demonstrations cannot usually be provided with the means available: reliability growth models, testing with stable reliability, structural dependability modelling, as well as more informal arguments based on good engineering practice. We state some rigorous arguments about the limits of what can be validated with each of such means. Combining evidence from these different sources would seem to raise the levels that can be validated; yet this improvement is not such as to solve the problem. It appears that engineering practice must take into account the fact that no solution exists, at present, for the validation of ultra-high dependability in systems relying on complex software.

Abstract- Measurement of software reliability by life testing involves executing the software on large numbers of test cases and recording the results. The number of failures observed is used to bound the failure probability even if the number of failures observed is zero. Typical analyses assume that all failures that occur are observed, but, in practice, failures occur without being observed. In this paper, we examine the effect of imperfect error detection, i.e., the situation in which a failure of the software may not be observed. If a conventional analysis associated with life testing is used, the confidence in the bound on the failure probability is optimistic. Our results show that imperfect error detection does not necessarily limit the ability of life testing to bound the probability of failure to the very low values required in critical systems. However, we show that the confidence level associated with a bound on failure probability cannot necessarily be made as high as desired, unless very strong assumptions are made about the error detection mechanism. Such assumptions are unlikely to be met in praetice, and so life testing is likely to be useful only for situations in which very high confidence levels are not required. Index Terms-Error detection, software reliability assessment, software testing, test oracles.

inherent uncertainty It has been said that the term software engineering is an aspiration not a description. We would like to be able to claim that we engineer software, in the same sense that we engineer an aero-engine, but most of us would agree that this is not currently an accurate description of our activities. My suspicion is that it never will be. From the point of view of this essay- i.e. dependability evaluation- a major difference between software and other engineering artefacts is that the former is pure design. Its unreliability is always the result of design faults, which in turn arise as a result of human intellectual failures. The unreliability of hardware systems, on the other hand, has tended until recently to be dominated by random physical failures of components- the consequences of the ‘perversity of nature’. Reliability theories have been developed over the years which have successfully allowed systems to be built to high reliability requirements, and the final system reliability to be evaluated accurately. Even for pure hardware systems, without software, however, the very success of these theories has more recently highlighted the importance of design faults in determining

This paper describes the specification-based testing and analysis tools, and associated processes, that were used to develop and certify safety-critical avionics systems in an industrial organization. These tools comprise an integrated development environment supporting specification acquisition and analysis, requirement-based automatic test vector generation, test coverage analysis, test driver generation, and test results analysis. The paper describes the specification model, method, development environment, and tool qualification approach. The capabilities of the automatic test generator are compared with foundational concepts and related testing strategies and mechanisms. 1.

It is a cruel reality that the goal of producing "perfect software " remains elusive. When software is part of a critical system, it is necessary to estimate the risk associated with its use. Software reliability is defined as the probability of failure free execution given a specific environment and a fixed time interval. The goal of reliability assessment is not just to estimate the failure probability of the program, `, but to gain the statistical confidence that ` is realistic. The transformational approach to software reliability assessment is a novel methodology which combines the strengths of formal verification and statistical testing in a unified and original reliability assessment framework. Program transformations and partial program proofs are used to amplify the effect of test cases; that is, they allow us to infer the behavior of the program on many inputs based on its behavior on one input. The main effect of the application of these transformations is the reduction in t...

Abstract not found