In this paper, we model the relation between testing effort, coverage and reliability. We present a logarithmic model that relates testing effort to test coverage (block, branch, c-use or p-use). The model is based on the hypothesis that the enumerables (like branches or blocks) for any coverage measure have different detectability, just like defects have different detectability. This model allows us to relate a test coverage measure directly with defect coverage. Data sets for programs with real defects are used to validate the model. The results are consistent with the known inclusion relationships among block, branch and p-use coverage measures. We show how defect density controls time to next failure. The model can eliminate the variables like test application strategy from consideration. It is suitable for high reliability applications where automatic (or manual) test generation is used to cover enumerables which have not yet been tested.

Statistical debugging uses lightweight instrumentation and statistical models to identify program behaviors that are strongly predictive of failure. However, most software is mostly correct; nearly all monitored behaviors are poor predictors of failure. We propose an adaptive monitoring strategy that mitigates the overhead associated with monitoring poor failure predictors. We begin by monitoring a small portion of the program, then automatically refine instrumentation over time to zero in on bugs. We formulate this approach as a search on the control-dependence graph of the program. We present and evaluate various heuristics that can be used for this search. We also discuss the construction of a binary instrumentor for incorporating the feedback loop into post-deployment monitoring. Performance measurements show that adaptive bug isolation yields an average performance overhead of 1 % for a class of large applications, as opposed to 87 % for realistic sampling-based instrumentation and 300 % for complete binary instrumentation.

"Software test-coverage measures" quantify the degree of thoroughness of testing. Tools are now available that measure test-coverage in terms of blocks, branches, computation-uses, predicate-uses, etc. that are covered. This paper models the relations among testing time, coverage, and reliability. An LE (logarithmic -exponential) model is presented that relates testing effort to test coverage (block, branch, computation-use, or predicate -use). The model is based on the hypothesis that the enumerable elements (like branches or blocks) for any coverage measure have various probabilities of being exercised; just like defects have various probabilities of being encountered. This model allows relating a test-coverage measure directly with defect-coverage. The model is fitted to 4 data-sets for programs with real defects. In the model, defect coverage can predict the time to next failure. The LE

We present a new approach that uses compilerdirected fault-injection for coverage testing of recovery code in Internet services to evaluate their robustness to operating system and I/O hardware faults. We define a set of program-fault coverage metrics that enable quantification of Java catch blocks exercised during fault-injection experiments. We use compiler analyses to instrument application code in two ways: to direct fault injection to occur at appropriate points during execution, and to measure the resulting coverage. As a proof of concept for these ideas, we have applied our techniques manually to Muffin, a proxy server; we obtained a high degree of coverage of catch blocks, with, on average, 85% of the expected faults per catch being experienced as caught exceptions.

this paper, we focus on methods of fault tolerance, and investigate the differences between hardware fault tolerance and software fault tolerance. 1.2 Fault, Error and Failure

This paper analyzes the effect of input profile selection on software testing using the concept of fault detectability profile. It shows that optimality of the input profile during testing depends on factors such as the planned testing effort and the error detectability profile. To achieve ultra-reliable software, selecting test input uniformly among different input domains is preferred. On the other hand, if testing effort is limited due to cost or schedule constraints, one should test only the highly used input domains. Use of operational profile is also needed for accurate determination of operational reliability.

In the statistical sampling method, as in any other statistical approaches for measuring software reliability, the inputs to the program are chosen according to the estimated probability with which they occur in field use, forming the operational profile. However, in practice it is very difficult to accurately assess the operational distribution of input points. Furthermore, a variety of factors can cause the operational distribution to change during field use making the estimation even more difficult. Musa has suggested that reducing the size of the input domain simplifies the task of determining operational profiles. In this paper, we present a class of techniques that reduce the dimensionality of input domains and describe their application. These techniques do not limit the functionality or change the input-output behavior of the program. An additional benefit of these techniques is the insensitivity of the reliability estimate to variations in the operational profile of variables ...

Testing a large-scale, real-life commercial software application is a very challenging task due to the constant changes in the software, the involvement of multiple programmers and testers, and a large amount of code. Integrating testing with development can help find program bugs at an earlier stage and hence reduce the overall cost. In this paper, we report our experience on how to apply eXVantage 1 (a tool suite for code coverage testing, debugging, performance profiling, etc.) to a large, complex Java application at the implementation and unit testing phases in Avaya. Our results suggest that programmers and testers can benefit from using eXVantage to monitor the testing process, gain confidence on the quality of their software, detect bugs which are otherwise difficult to reveal, and identify performance bottlenecks in terms of which part of code is most frequently executed. KEY WORDS Code coverage, program debugging, performance profiling, execution slicing, eXVantage, JBT 1.