## Testing, abstraction, theorem proving: better together (2006)

Venue: | In Software Testing and Analysis (ISSTA |

Citations: | 16 - 0 self |

### BibTeX

@INPROCEEDINGS{Yorsh06testing,abstraction,,

author = {Greta Yorsh},

title = {Testing, abstraction, theorem proving: better together},

booktitle = {In Software Testing and Analysis (ISSTA},

year = {2006},

pages = {145--156},

publisher = {ACM}

}

### OpenURL

### Abstract

We present a method for static program analysis that leverages tests and concrete program executions. State abstractions generalize the set of program states obtained from concrete executions. A theorem prover then checks that the generalized set of concrete states covers all potential executions and satisfies additional safety properties. Our method finds the same potential errors as the mostprecise abstract interpreter for a given abstraction and is potentially more efficient. Additionally, it provides a new way to tune the performance of the analysis by alternating between concrete execution and theorem proving. We have implemented our technique in a prototype for checking properties of C # programs.

### Citations

1882 | Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints - COUSOT, COUSOT - 1977 |

623 | Model Checking and Abstraction
- Clarke, Grumberg, et al.
- 1994
(Show Context)
Citation Context ...abstraction to overapproximate the reachable concrete states of a program. Abstraction and abstract interpretation [6] are key tools for automatically proving properties of systems, both for hardware =-=[5, 9]-=- and software systems [27]. An abstraction function α maps concrete program states to abstract states. An abstract state is reachable if it is the abstraction of some reachable concrete state. Identif... |

599 | Construction of abstract state graphs with PVS
- Graf, Saïdi
- 1997
(Show Context)
Citation Context ...s .NET managed assemblies, and provides means for analyzing, rewriting, and executing programs. Our implementation takes advantage of all these features. Our implementation uses predicate abstraction =-=[17]-=- (without refinement), and supports user-defined predicates. It also can automatically generate a default set of predicates by a backwards dataflow analysis from the conditional branches that infers t... |

573 | Automatic discovery of linear restraints among variables of a program, in: POPL
- Cousot, Halbwachs
- 1978
(Show Context)
Citation Context ...sents all concrete states reachable from I (but possibly other states): LFPI(f) ⊆ γ(a). (3)s[1]procedure basic(T0) [2] a := ⊥ [3] T := T0 [4] while(true) begin [5] C := Execute(f,T) [6] a := a ⊔ α(C) =-=[7]-=- if exists σ ∈ f(γ(a)) s.t. σ /∈ γ(a) [8] then T := {σ} [9] else return a [10] end Figure 3: The basic procedure. Here, T0, T, C ⊆ C, and a ∈ A. If α(T0) = α(I), then the result of the procedure is a ... |

563 |
Assigning meanings to programs
- Floyd
- 1967
(Show Context)
Citation Context ...sed. of program points, called cutpoints, where the abstraction is computed. As in deductive verification, a minimal set of cutpoints is a set which cuts every cycle in a program’s control flow graph =-=[15]-=-. The runtime overhead of computing abstract state coverage decreases when there are fewer cutpoints. Furthermore, having fewer cutpoints potentially improves the precision of our method, as the abstr... |

543 | Dynamically discovering likely program invariants to support program evolution
- Ernst, Cockrell, et al.
- 2001
(Show Context)
Citation Context ...m. The method of [33] can be described by replacing C := execute(f,T) with C := T in line[5] of Fig. 3. Combining Dynamic and Static Analyses. Daikon uses dynamic analysis to detect likely invariants =-=[14]-=-. It executes the program on a test set, examining the values of the concrete states, and detects patterns and relationships among those values. It reports properties that hold over execution of the g... |

538 | Parametric shape analysis via 3-valued logic
- Sagiv, Reps, et al.
(Show Context)
Citation Context ...r method is applicable beyond predicate abstraction. We have implemented another prototype, based on the TVLA system [24]. The TVLA system performs abstract interpretation using canonical abstraction =-=[31]-=-, and supports reasoning about recursive datastructures. We have implemented a special-purpose model generator that uses canonical abstraction to guide the search for models. For concrete execution, w... |

527 | Principles of Program Analysis
- Nielson, Nielson, et al.
- 1999
(Show Context)
Citation Context ...te the reachable concrete states of a program. Abstraction and abstract interpretation [6] are key tools for automatically proving properties of systems, both for hardware [5, 9] and software systems =-=[27]-=-. An abstraction function α maps concrete program states to abstract states. An abstract state is reachable if it is the abstraction of some reachable concrete state. Identifying exactly the reachable... |

517 | Dart: directed automated random testing
- Godefroid, Klarlund, et al.
- 2005
(Show Context)
Citation Context ...t injection, fabricated states, adequacy criteria, coverage, state-based coverage 1. INTRODUCTION Recently, there has been much interest in combining dynamic and static methods for analyzing programs =-=[28, 16, 8, 29]-=-. Dynamic analysis (or testing) is based on concrete program executions and underapproximates the set of program behaviors. That is, if BP denotes the set of all behaviors of a program P then dynamic ... |

463 | model checking programs
- Visser, Havelund, et al.
(Show Context)
Citation Context ...verage; (2) symbolic execution of loop-free code fragments to compute SP; (3) state manipulation to create fabricated states. Explicit-state model checkers such as SPIN [21], CMC [26], JavaPathFinder =-=[34]-=-, XRT [18], which perform systematic and exhaustive testing, provide a good starting point (though not all support symbolic execution). A model checker analyzes several executions of the program at on... |

441 |
The SPIN Model Checker: Primer and Reference Manual
- Holzmann
- 2004
(Show Context)
Citation Context ...ns to compute abstract state coverage; (2) symbolic execution of loop-free code fragments to compute SP; (3) state manipulation to create fabricated states. Explicit-state model checkers such as SPIN =-=[21]-=-, CMC [26], JavaPathFinder [34], XRT [18], which perform systematic and exhaustive testing, provide a good starting point (though not all support symbolic execution). A model checker analyzes several ... |

393 | S.K.: Automatic predicate abstraction of c programs
- Ball, Majumdar, et al.
- 2001
(Show Context)
Citation Context ...s an error on a small list, up to size 7. 6. RELATED WORK Automated Construction of Abstract Transformers. Theorem provers have been used for the automated construction of abstract transition systems =-=[2, 20, 36, 33]-=-, especially in parametric abstract domains, such as predicate abstraction [17] and canonical abstraction [31], where the abstraction is defined per-program. In many cases, an exponential number of th... |

352 | Simplify: A theorem prover for program checking
- Detlefs, Nelson, et al.
- 2003
(Show Context)
Citation Context ...ating shape analyses [24] and the XRT system for generating unit tests [18]. The XRT implementation supports all C# features including pointers and procedures. It employs the Simplify theorem prover =-=[10]-=- and demonstrates the feasibility of our approach. • Our method provides an effective test for abstraction-based adequacy of a test set, as defined by [12, 1]. • We show that our method can find safet... |

296 | CUTE: a concolic unit testing engine for C
- Sen, Marinov, et al.
- 2005
(Show Context)
Citation Context ...xpoint computation where the Daikon-ESC/Java two-step process may fail to find a proof. Recent work combines random test generation and concrete execution with symbolic execution and model generation =-=[16, 32, 8]-=-. These methods use symbolic techniques to direct the generation of tests towards unexplored paths in order to find errors faster. However, these methods do not use abstraction, and in general cannot ... |

257 | Software Unit Test Coverage and Adequacy
- Zhu, Hall, et al.
- 1997
(Show Context)
Citation Context ...ecause our method using fabricated states may cover abstract states that are not reachable (but required for a proof). Test Adequacy. In contrast to the traditional white-box adequacy criteria (e.g., =-=[37]-=-), we choose an abstraction based on the property of interest, and then define adequacy with respect to the abstraction. When used with a powerset abstraction, our adequacy requirement appears to be a... |

184 | CMC: A Pragmatic Approach to Model Checking Real Code
- Musuvathi, Park, et al.
(Show Context)
Citation Context ...ute abstract state coverage; (2) symbolic execution of loop-free code fragments to compute SP; (3) state manipulation to create fabricated states. Explicit-state model checkers such as SPIN [21], CMC =-=[26]-=-, JavaPathFinder [34], XRT [18], which perform systematic and exhaustive testing, provide a good starting point (though not all support symbolic execution). A model checker analyzes several executions... |

142 | Software verification with BLAST
- Henzinger, Jhala, et al.
- 2003
(Show Context)
Citation Context ...s an error on a small list, up to size 7. 6. RELATED WORK Automated Construction of Abstract Transformers. Theorem provers have been used for the automated construction of abstract transition systems =-=[2, 20, 36, 33]-=-, especially in parametric abstract domains, such as predicate abstraction [17] and canonical abstraction [31], where the abstraction is defined per-program. In many cases, an exponential number of th... |

112 | TVLA: A system for implementing static analyses
- Lev-Ami, Sagiv
- 2000
(Show Context)
Citation Context ...ound and as precise as the result of the most-precise abstract interpreter (over the same abstract domain). • We implemented our method in two platforms: the TVLA system for generating shape analyses =-=[24]-=- and the XRT system for generating unit tests [18]. The XRT implementation supports all C# features including pointers and procedures. It employs the Simplify theorem prover [10] and demonstrates the... |

90 |
ESC/Java user’s manual
- Leino, Nelson, et al.
- 2000
(Show Context)
Citation Context ...those values. It reports properties that hold over execution of the given test set, but not necessary over all program executions. In [28], likely invariants produced by Diakon are used with ESC/Java =-=[23]-=- verificaprocedure fabricated abstract maximal description states states length search 2 21 5 searches a list for an element with a specified value reverse 4 57 6 reverses a singly-linked list in-sit... |

86 | The model evolution calculus
- Baumgartner, Tinelli
- 2003
(Show Context)
Citation Context ...I(f). An abstract value a ∈ A is a sound overapproximation of P if a represents all concrete states reachable from I (but possibly other states): LFPI(f) ⊆ γ(a). (3)s[1]procedure basic(T0) [2] a := ⊥ =-=[3]-=- T := T0 [4] while(true) begin [5] C := Execute(f,T) [6] a := a ⊔ α(C) [7] if exists σ ∈ f(γ(a)) s.t. σ /∈ γ(a) [8] then T := {σ} [9] else return a [10] end Figure 3: The basic procedure. Here, T0, T,... |

72 | Testing: A Road Map
- Harrold
- 2000
(Show Context)
Citation Context ...rion implies the adequacy criterion we defined in Section 1. Our algorithm provides an effective way to check adequacy of a given test set. 7. CONCLUSIONS Our method can be viewed as bridging the gap =-=[19, 13]-=- between testing and verification. Our method finds the same potential errors as the most-precise abstract interpreter for a given abstraction. Additionally, it provides a new way to tune performance ... |

64 | Check ’n’ Crash: combining static checking and testing
- Csallner, Smaragdakis
- 2005
(Show Context)
Citation Context ...ve mutual exclusion (without any abstraction refinement), and the state space has only 17 abstract states.s[1]n:=0; [2]while(*) { [3] if (*) { [4] n := n+1; [5] } else { [6] if(n==500) [7] assert(0); =-=[8]-=- n:=0; [9] } [10]} (a) (b) Figure 6: (a) Example program, (b) Abstract state-space. 5.3 Hybrid Approach For certain programs, concrete execution might go on for a long time without covering a new abst... |

62 | New techniques that improve mace-style finite model building - Claessen, Sörensson - 2003 |

58 | Invariant Inference for Static Checking: An Empirical Evaluation
- Nimmer, Ernst
- 2002
(Show Context)
Citation Context ...t injection, fabricated states, adequacy criteria, coverage, state-based coverage 1. INTRODUCTION Recently, there has been much interest in combining dynamic and static methods for analyzing programs =-=[28, 16, 8, 29]-=-. Dynamic analysis (or testing) is based on concrete program executions and underapproximates the set of program behaviors. That is, if BP denotes the set of all behaviors of a program P then dynamic ... |

54 | A theory of predicate-complete test coverage and generation
- Ball
- 2005
(Show Context)
Citation Context ...ures. It employs the Simplify theorem prover [10] and demonstrates the feasibility of our approach. • Our method provides an effective test for abstraction-based adequacy of a test set, as defined by =-=[12, 1]-=-. • We show that our method can find safety proofs with much simpler abstractions than those used by [29], which uses a combination of concrete execution, abstraction and theorem proving to find bisim... |

51 | Symbolically computing most-precise abstract operations for shape analysis
- Yorsh, Reps, et al.
- 2004
(Show Context)
Citation Context ...s an error on a small list, up to size 7. 6. RELATED WORK Automated Construction of Abstract Transformers. Theorem provers have been used for the automated construction of abstract transition systems =-=[2, 20, 36, 33]-=-, especially in parametric abstract domains, such as predicate abstraction [17] and canonical abstraction [31], where the abstraction is defined per-program. In many cases, an exponential number of th... |

37 | Vampire 1.1 (system description - Riazanov, Voronkov - 2001 |

32 | Symbolic implementation of the best transformer
- Reps, Sagiv, et al.
- 2004
(Show Context)
Citation Context |

25 |
Concrete model checking with abstract matching and refinement
- Pasareanu, Pelánek, et al.
- 2005
(Show Context)
Citation Context ...t injection, fabricated states, adequacy criteria, coverage, state-based coverage 1. INTRODUCTION Recently, there has been much interest in combining dynamic and static methods for analyzing programs =-=[28, 16, 8, 29]-=-. Dynamic analysis (or testing) is based on concrete program executions and underapproximates the set of program behaviors. That is, if BP denotes the set of all behaviors of a program P then dynamic ... |

24 |
XRT - Exploring Runtime for .NET - Architecture and Applications
- Grieskamp, Tillmann, et al.
- 2005
(Show Context)
Citation Context ...se abstract interpreter (over the same abstract domain). • We implemented our method in two platforms: the TVLA system for generating shape analyses [24] and the XRT system for generating unit tests =-=[18]-=-. The XRT implementation supports all C# features including pointers and procedures. It employs the Simplify theorem prover [10] and demonstrates the feasibility of our approach. • Our method provides... |

21 |
Online minimization of transition systems (extended abstract
- LEE, M
- 1992
(Show Context)
Citation Context ... be used in unit-test generation. 5.2 Avoiding Unnecessary Abstraction Refinement We are not the first to demonstrate that concrete execution plus abstraction can be used to verify program properties =-=[22, 29, 1]-=-. However, previous work in the area required much stronger abstractions than necessary to verify the safety properties of interest. One approach is to find an abstract system that is bisimilar to the... |

6 |
Abstract Interpretation and Partial Refinement for Model Checking
- Dams
- 1996
(Show Context)
Citation Context ...abstraction to overapproximate the reachable concrete states of a program. Abstraction and abstract interpretation [6] are key tools for automatically proving properties of systems, both for hardware =-=[5, 9]-=- and software systems [27]. An abstraction function α maps concrete program states to abstract states. An abstract state is reachable if it is the abstraction of some reachable concrete state. Identif... |

5 | A Discipline of Programming - Dijksta - 1976 |

5 | SPASS: An automated theorem prover for first-order logic with equality. Available at “http://spass.mpi-sb.mpg.de/index.html
- Weidenbach
(Show Context)
Citation Context ...heorem prover and a model generator which can generate a concrete counterexample to validity of a formula. Technically, there are off-the-shelf automatic theorem provers that can be used, e.g., SPASS =-=[35]-=-, Vampire [30], Simplify [10]. Unfortunately, most such theorem provers do not produce concrete counterexamples for invalid formulas (with the exception of Darwin [3]). Instead, a separate tool for mo... |

3 |
Generating concrete counter examples for arbitrary abstract domains. Unpublished Manuscript
- Erez, Sagiv, et al.
- 2003
(Show Context)
Citation Context ...ures. It employs the Simplify theorem prover [10] and demonstrates the feasibility of our approach. • Our method provides an effective test for abstraction-based adequacy of a test set, as defined by =-=[12, 1]-=-. • We show that our method can find safety proofs with much simpler abstractions than those used by [29], which uses a combination of concrete execution, abstraction and theorem proving to find bisim... |

2 |
Modular Static Analysis of Object Oriented Languages
- Logozzo
- 2004
(Show Context)
Citation Context ...ysis, the abstract states on the entry and exit of all methods are the same. This set of abstract states, in fact, represents the class invariants, under certain conditions about the class, stated in =-=[25]-=-. The analysis can output the inferred class invariants in the form of logical formulas, by computing�γ of the relevant abstract states. Note that, as we are using state-based abstractions, our approa... |

1 |
Static and dynamic analysis: Synergy and
- Ernst
- 2003
(Show Context)
Citation Context ...rion implies the adequacy criterion we defined in Section 1. Our algorithm provides an effective way to check adequacy of a given test set. 7. CONCLUSIONS Our method can be viewed as bridging the gap =-=[19, 13]-=- between testing and verification. Our method finds the same potential errors as the most-precise abstract interpreter for a given abstraction. Additionally, it provides a new way to tune performance ... |