Explicitly stated program invariants can help programmers by identifying program properties that must be preserved when modifying code. In practice, however, these invari-ants are usually implicit. An alternative to expecting pro-grammers to fully annotate code with invariants is to au-tomatically infer invariants from the program itself. This research focuses on dynamic techniques for discovering in-variants from execution traces. This paper reports two results. First, it describes techniques for dynamically discovering invariants, along with an instru-menter and an inference engine that embody these tech-niques. Second, it reports on the application of the engine to two sets of target programs. In programs from Gries’s work on program derivation, we rediscovered predefined in-variants. In a C program lacking explicit invariants, we dis-covered invariants that assisted a software evolution task.

Abstract not found

We experimentally evaluate the effectiveness of using cluster analysis of execution profiles to find failures among the executions induced by a set of potential test cases. We compare several filtering procedures for selecting executions to evaluate for conformance to requirements. Each filtering procedure involves a choice of a sampling strategy and a clustering metric. The results suggest that filtering procedures based on clustering are more effective than simple random sampling for identifying failures in populations of operational executions, with adaptive sampling from clusters being the most effective sampling strategy. The results also suggest that clustering metrics that give extra weight to unusual profile features are most effective. Scatter plots of execution populations, produced by multidimensional scaling, are used to provide intuition for these results. Keywords: Observation-based testing, software testing, operational testing, beta testing, cluster analysis, multidimensional scaling 1.

Producing specifications by dynamic (runtime) analysis of program executions is potentially unsound, because the analyzed executions may not fully characterize all possible executions of the program. In practice, how accurate are the results of a dynamic analysis? This paper describes the results of an investigation into this question, determining how much specifications generalized from program runs must be changed in order to be verified by a static checker.

This paper shows how to integrate two complementary techniques for manipulating program invariants: dynamic detection and static verification. Dynamic detection proposes likely invariants based on program executions, but the resulting properties are not guaranteed to be true over all possible executions. Static verification checks that properties are always true, but it can be difficult and tedious to select a goal and to annotate programs for input to a static checker. Combining these techniques overcomes the weaknesses of each: dynamically detected invariants can annotate a program or provide goals for static verification, and static veri cation can confirm properties proposed by a dynamic tool. We have

Abstract. A common method to localize defects is to compare the coverage of passing and failing program runs: A method executed only in failing runs, for instance, is likely to point to the defect. Some failures, though, come to be only through a specific sequence of method calls, such as multiple deallocation of the same resource. Such sequences can be collected from arbitrary Java programs at low cost; comparing object-specific sequences predicts defects better than simply comparing coverage. In a controlled experiment, our technique pinpointed the defective class in 36 % of all test runs. 1

We describe the design of jRapture: a tool for capturing and replaying Java ™ ™ program executions in the field. jRapture works with Java binaries (byte code) and any compliant implementation of the Java virtual machine. It employs a lightweight, transparent capture process that permits unobtrusive capture of a Java program’s executions. jRapture captures interactions between a Java program and the system, including GUI, file, and console inputs, among other types, and on replay it presents each thread with exactly the same input sequence it saw during capture. In addition, jRapture has a profiling interface that permits a Java program to be instrumented for profiling ⎯ after its executions have been captured. Using an XML-based profiling specification language a tester can specify various forms of profiling to be carried out during replay.

We present a general method for fault localization based on abstracting over program traces, and a tool that implements the method using Ernst's notion of potential invariants. Our experiments so far have been unsatisfactory, suggesting that further research is needed before invariants can be used to locate faults.

Observation-based testing calls for analyzing profiles of executions induced by potential test cases, in order to select a subset of executions to be checked for conformance to requirements. A family of techniques for selecting such a subset is evaluated experimentally. These techniques employ automatic cluster analysis to partition executions, and they use various sampling techniques to select executions from clusters. The experimental results support the hypothesis that with appropriate profiling, failures often have unusual profiles that are revealed by cluster analysis. The results also suggest that failures often form small clusters or chains in sparsely-populated areas of the profile space. A form of adaptive sampling called failure-pursuit sampling is proposed for revealing failures in such regions, and this sampling method is evaluated experimentally. The results suggest that failure-pursuit sampling is effective.

Analyzing the executions of a buggy software program is essentially a data mining process. Although many interesting methods have been developed to trace crashing bugs (such as memory violation and core dumps), it is still difficult to analyze noncrashing bugs (such as logical errors). In this paper, we develop a novel method to classify the structured traces of program executions using software behavior graphs. By analyzing the correct and incorrect executions, we have made good progress at the isolation of program regions that may lead to the faulty executions. The classification framework is built on an integration of closed graph mining and SVM classification. More interestingly, suspicious regions are identified through the capture of the classification accuracy change, which is measured incrementally during program execution. Our performance study and case-based experiments show that our approach is both effective and efficient.