Producing specifications by dynamic (runtime) analysis of program executions is potentially unsound, because the analyzed executions may not fully characterize all possible executions of the program. In practice, how accurate are the results of a dynamic analysis? This paper describes the results of an investigation into this question, determining how much specifications generalized from program runs must be changed in order to be verified by a static checker.

Dynamic detection of likely invariants is a program analysis that generalizes over observed values to hypothesize program properties. The reported program properties are a set of likely invariants over the program, also known as an operational abstraction. Operational abstractions are useful in testing, verification, bug detection, refactoring, comparing behavior, and many other tasks. Previous techniques for...

Producing specifications by dynamic (runtime) analysis of program executions is potentially unsound, because the analyzed executions may not fully characterize all possible executions of the program. In practice, how accurate are the results of a dynamic analysis? This paper describes the results of an investigation into this question, comparing specifications generalized from program runs with specifications verified by a static checker. The surprising result is that for a collection of modest programs, small test suites captured all or nearly all program behavior necessary for a specific type of static checking, permitting the inference and verification of useful specifications. For ten programs of 100--800 lines, the average precision, a measure of correctness, was .95 and the average recall, a measure of completeness, was .94. This is a

In the long run, programs should be written from the start with verification in mind. Programs written in such a way are likely to be much easier to verify. They will avoid hard-to-verify features, may have better designs, will be accompanied by full formal specifications, and may be annotated with verification information. However, even if programs should be written this way, not all of them will. In the short run, it is crucial to verify the legacy programs that make up our existing computing infrastructure, and to provide tools that assist programmers in performing verification tasks and — equally importantly — in shifting their mindset to one of program verification. I propose approaches to verification that may assist in reaching these goals. The key idea underlying the approaches is specification inference (Section 1). This is a machine learning technique that produces, from an existing program, a (likely) specification of that program. Specifications are very frequently missing from real-world programs, but are required for verification. The inferred specification can serve as a goal for verification. I discuss three different approaches that can use such inferred specifications. One uses a heavyweight proof assistant (Section 2), one uses an automated theorem prover (Section 3, and one requires no user interaction but provides no guarantee (Section 4). 1 Inference of likely specifications