## Mechanized metatheory for the masses: The POPLmark challenge (2005)

### Cached

### Download Links

- [www.cis.upenn.edu]
- [research.microsoft.com]
- [www.cis.upenn.edu]
- [www.cis.upenn.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In Theorem Proving in Higher Order Logics: 18th International Conference, number 3603 in LNCS |

Citations: | 144 - 14 self |

### BibTeX

@INPROCEEDINGS{Aydemir05mechanizedmetatheory,

author = {Brian E. Aydemir and Aaron Bohannon and Matthew Fairbairn and J. Nathan Foster and Benjamin C. Pierce and Peter Sewell and Dimitrios Vytiniotis and Stephanie Weirich and Steve Zdancewic},

title = {Mechanized metatheory for the masses: The POPLmark challenge},

booktitle = {In Theorem Proving in Higher Order Logics: 18th International Conference, number 3603 in LNCS},

year = {2005},

pages = {50--65},

publisher = {Springer-Verlag}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. How close are we to a world where every paper on programming languages is accompanied by an electronic appendix with machinechecked proofs? We propose an initial set of benchmarks for measuring progress in this area. Based on the metatheory of System F<:, a typed lambda-calculus with second-order polymorphism, subtyping, and records, these benchmarks embody many aspects of programming languages that are challenging to formalize: variable binding at both the term and type levels, syntactic forms with variable numbers of components (including binders), and proofs demanding complex induction principles. We hope that these benchmarks will help clarify the current state of the art, provide a basis for comparing competing technologies, and motivate further research. 1

### Citations

753 |
Types and Programming Languages
- Pierce
- 2002
(Show Context)
Citation Context ...lation, refining universal quantifiers to carry subtyping constraints,sand adding records, record subtyping, and record patterns. Our presentation is based on Pierce’s Types and Programming Languages =-=[29]-=-. The challenge comprises three distinct parts. The first deals just with the type language of F<:; the second considers terms, evaluation, and type soundness. Each of these is further subdivided into... |

601 |
Assigning meanings to programs
- Floyd
(Show Context)
Citation Context ...nd to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems [31, 12, 5, 15, 9, 21] up to the grand challenges of Floyd, Hoare, and Moore =-=[7, 14, 20]-=-. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-value variant of System F<: [3], enriched with records, record subt... |

565 | A syntactic approach to type soundness
- Wright, Felleisen
- 1994
(Show Context)
Citation Context ... of formalization when they include binders, though unfortunately there are no examples of this in call-by-value F<:. Type soundness is usually proven in the style popularized by Wright and Felleisen =-=[37]-=-, in terms of preservation and progress theorems. Challenge 2A is to prove these properties for pure F<:. 3.3 Theorem [Preservation]: If Γ ⊢ t : T and t −→ t ′ , then Γ ⊢ t ′ : T. ✷ 3.4 Theorem [Progr... |

311 |
Higher-order abstract syntax
- Pfenning, Elliott
(Show Context)
Citation Context ...ariables are ordinary names while bound variables are represented using de Bruijn indices [18]. A radically different approach to representing terms with binders is higherorder abstract syntax (HOAS) =-=[28]-=-. In HOAS representations, binders in the meta-language are used to represent binders in the object language. Our experience with HOAS encodings (mainly as realized in Twelf) is that they provide a co... |

237 | Foundational proof-carrying code
- Appel
(Show Context)
Citation Context ...], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code =-=[1]-=- and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mechanized metatheory more accessible to programming languages researchers. We hope to stimulate progress by providing ... |

174 | Nominal logic, a first order theory of names and binding
- Pitts
(Show Context)
Citation Context ... of reasoning about binders based upon a set theory extended with an intrinsic notion of permutation [8]. Pitts followed this up by proposing a new “nominal” logic based upon the idea of permutations =-=[30]-=-. More recent work by Urban proposes methods based on the same intuitions but carried out within a conventional logic [33]. Our own preliminary experiments with Urban’s methods have been encouraging. ... |

153 | A new approach to abstract syntax involving binders
- Gabbay, Pitts
- 1999
(Show Context)
Citation Context ... “swapping” as a primitive, and use it to build a nominal logic. Gabbay and Pitts proposed a method of reasoning about binders based upon a set theory extended with an intrinsic notion of permutation =-=[8]-=-. Pitts followed this up by proposing a new “nominal” logic based upon the idea of permutations [30]. More recent work by Urban proposes methods based on the same intuitions but carried out within a c... |

115 | An Extension of System F with Subtyping
- Cardelli, Martini, et al.
- 1994
(Show Context)
Citation Context ...tegies, and related tools. Collectively, we have applied automated theorem proving technology to a number of problems, including proving transitivity of the algorithmic subtype relation in System F<: =-=[3]-=-, proving type soundness of Featherweight Java, proving type soundness of variants of the simply typed λ-calculus and F<:, and a substantial formalization of the behavior of TCP, UDP, and the Sockets ... |

106 | A machine-checked model for a Java-like language, virtual machine, and compiler
- Klein, Nipkow
(Show Context)
Citation Context ...tness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java =-=[24, 17, 23]-=-. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mec... |

102 |
CSPLib: A Benchmark Library for Constraints
- Gent, Walsh
- 1999
(Show Context)
Citation Context ...t within the theorem proving community to focus attention on specific areas and to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems =-=[31, 12, 5, 15, 9, 21]-=- up to the grand challenges of Floyd, Hoare, and Moore [7, 14, 20]. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-v... |

102 | The TPTP Problem Library
- Suttner, Sutcliffe
- 2002
(Show Context)
Citation Context ...t within the theorem proving community to focus attention on specific areas and to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems =-=[31, 12, 5, 15, 9, 21]-=- up to the grand challenges of Floyd, Hoare, and Moore [7, 14, 20]. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-v... |

93 | Toward a foundational typed assembly language
- Crary
(Show Context)
Citation Context ...rectness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages =-=[4]-=-. Inspired by these successes, we seek to make mechanized metatheory more accessible to programming languages researchers. We hope to stimulate progress by providing a framework for comparing alternat... |

92 | R.: The Verifying Compiler: a Grand Challenge for Computer Research
- Hoare
- 2003
(Show Context)
Citation Context ...nd to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems [31, 12, 5, 15, 9, 21] up to the grand challenges of Floyd, Hoare, and Moore =-=[7, 14, 20]-=-. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-value variant of System F<: [3], enriched with records, record subt... |

67 | C Formalised in HOL
- Norrish
- 1998
(Show Context)
Citation Context ...rmalization of languages is already within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C =-=[25]-=-, semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of mechanized metatheory in... |

57 | Some lambda calculus and type theory formalized
- McKinna, Pollack
- 1999
(Show Context)
Citation Context ... Pollack use a hybrid approach that combines the above two representation strategies. In this approach, free variables are ordinary names while bound variables are represented using de Bruijn indices =-=[18]-=-. A radically different approach to representing terms with binders is higherorder abstract syntax (HOAS) [28]. In HOAS representations, binders in the meta-language are used to represent binders in t... |

53 | Five axioms of alpha conversion
- Gordon, Melham
- 1996
(Show Context)
Citation Context ...ometimes require significant ingenuity to encode particular language features or proof ideas in this style. Gordon and Melham propose a way to axiomatize inductive reasoning over untyped lambda-terms =-=[11]-=- and suggest that other inductive structures with binding can be encoded by setting up a correspondence with the untyped lambda terms. Norrish has pursued this direction [26, 27], but observes that th... |

40 |
µJava: Embedding a programming language in a theorem prover
- Nipkow, Oheimb, et al.
- 2000
(Show Context)
Citation Context ...tness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java =-=[24, 17, 23]-=-. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mec... |

39 |
Proof Theoretic Studies about a Minimal Type System Integrating Inclusion and Parametric Polymorphism
- Ghelli
- 1990
(Show Context)
Citation Context ... rules Γ ⊢ S <: T {li i∈1..n } ⊆ {kj j∈1..m } if kj = li, then Γ ⊢ Sj <: Ti Γ ⊢ {kj:Sj j∈1..m } <: {li:Ti i∈1..n } (SA-Rcd) Although it has been shown that records can actually be encoded in pure F<: =-=[2, 10]-=-, dealing with them directly is a worthwhile task since, unlike other syntactic forms, record types have an arbitrary (finite) number of fields. Also, the informal proof for Challenge 1A extends to re... |

29 | A grand challenge proposal for formal methods: A verified stack
- Moore
(Show Context)
Citation Context ...nd to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems [31, 12, 5, 15, 9, 21] up to the grand challenges of Floyd, Hoare, and Moore =-=[7, 14, 20]-=-. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-value variant of System F<: [3], enriched with records, record subt... |

25 |
The Definition of Standard ML, Revised Edition
- Milner, Tofte, et al.
- 1997
(Show Context)
Citation Context ...of assistant. In some cases one could use a definition directly as a prototype. 3. Support for engineering large-scale definitions. As we move to full language definitions—on the scale of Standard ML =-=[19]-=- or larger—pragmatic “software engineering” issues become increasingly important, as do the potential benefits of tool support. For large definitions, the need for elegant and concise notation becomes... |

23 | The apprentice challenge
- Moore, Porter
(Show Context)
Citation Context ...t within the theorem proving community to focus attention on specific areas and to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems =-=[31, 12, 5, 15, 9, 21]-=- up to the grand challenges of Floyd, Hoare, and Moore [7, 14, 20]. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-v... |

19 |
Extensible records in a pure calculus of subtyping', Research Report 81, DEC Systems Research Center, January. Reprinted in: Theoretical Aspects of Object-Oriented Programming
- Cardelli
- 1992
(Show Context)
Citation Context ... rules Γ ⊢ S <: T {li i∈1..n } ⊆ {kj j∈1..m } if kj = li, then Γ ⊢ Sj <: Ti Γ ⊢ {kj:Sj j∈1..m } <: {li:Ti i∈1..n } (SA-Rcd) Although it has been shown that records can actually be encoded in pure F<: =-=[2, 10]-=-, dealing with them directly is a worthwhile task since, unlike other syntactic forms, record types have an arbitrary (finite) number of fields. Also, the informal proof for Challenge 1A extends to re... |

19 | Reasoning with the formal definition of Standard ML
- Syme
- 1993
(Show Context)
Citation Context ...ady within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML =-=[32, 34, 13]-=-, and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [... |

16 | Type inference verified: Algorithm W in Isabelle/HOL
- Naraschewski, Nipkow
- 1999
(Show Context)
Citation Context ...ions, etc.sLarge scale formalization of languages is already within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML =-=[6, 22]-=-, semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of me... |

14 | Recursive function definition for types with binders
- Norrish
- 2004
(Show Context)
Citation Context ...over untyped lambda-terms [11] and suggest that other inductive structures with binding can be encoded by setting up a correspondence with the untyped lambda terms. Norrish has pursued this direction =-=[26, 27]-=-, but observes that these axioms are cumbersome to use without some assistance from the theorem-proving tool. In particular, the axioms use universal quantification in inductive hypotheses where in in... |

14 |
R.: A formalised first-order confluence proof for the λ-calculus using one-sorted variable names
- Brotherston, Vestergaard
- 2003
(Show Context)
Citation Context ...eting the challenge. A first-order, named approach very similar in flavor to standard informal presentations was used by Vestergaard and Brotherston to formalize some metatheory of untyped λ-calculus =-=[35, 36]-=-. Their representation requires that each binder initially be assigned a unique name—one aspect of the so-called Barendregt convention. Another popular concrete representation is de Bruijn’s nameless ... |

14 | Java: Embedding a programming language in a theorem prover
- Nipkow, Oheimb, et al.
- 2000
(Show Context)
Citation Context ...tness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java =-=[24, 17, 23]-=-. Some other signicant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mech... |

13 |
Oheimb. Java light is type-safe—definitely
- Nipkow, von
- 1998
(Show Context)
Citation Context ...tness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java =-=[24, 17, 23]-=-. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mec... |

10 | Certification of a type inference tool for ML: DamasMilner within Coq
- Dubois, Ménissier-Morain
(Show Context)
Citation Context ...ions, etc.sLarge scale formalization of languages is already within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML =-=[6, 22]-=-, semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of me... |

8 | The mechanisation of Barendregt-style equational proofs (the residual perspective
- Vestergaard, Brotherston
- 2001
(Show Context)
Citation Context ...eting the challenge. A first-order, named approach very similar in flavor to standard informal presentations was used by Vestergaard and Brotherston to formalize some metatheory of untyped λ-calculus =-=[35, 36]-=-. Their representation requires that each binder initially be assigned a unique name—one aspect of the so-called Barendregt convention. Another popular concrete representation is de Bruijn’s nameless ... |

6 |
Mechanising Hankin and Barendregt using the Gordon-Melham axioms
- Norrish
- 2003
(Show Context)
Citation Context ...over untyped lambda-terms [11] and suggest that other inductive structures with binding can be encoded by setting up a correspondence with the untyped lambda terms. Norrish has pursued this direction =-=[26, 27]-=-, but observes that these axioms are cumbersome to use without some assistance from the theorem-proving tool. In particular, the axioms use universal quantification in inductive hypotheses where in in... |

5 |
Studying the ML module system
- Gunter, Maharaj
- 1995
(Show Context)
Citation Context ...ady within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML =-=[32, 34, 13]-=-, and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other significant existing applications of mechanized metatheory include Foundational Proof Carrying Code [... |

3 |
Inductive challenge problems
- Dennis
- 2000
(Show Context)
Citation Context |

3 |
The dream corpus of inductive conjectures
- Green
- 1999
(Show Context)
Citation Context |

3 |
Certi of a type inference tool for ML: Damas-Milner within Coq
- Dubois, Menissier-Morain
- 1999
(Show Context)
Citation Context ...ions, etc. Large scale formalization of languages is already within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML =-=[6, 22]-=-, semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other signicant existing applications of mec... |

2 |
Nominal techniques in Isabelle/HOL. Accepted at CADE-20 in Tallinn. See http://www.mathematik.uni-muenchen.de/ ∼urban/nominal/. 34. Myra VanInwegen and Elsa
- Urban, Tasson
(Show Context)
Citation Context ...his up by proposing a new “nominal” logic based upon the idea of permutations [30]. More recent work by Urban proposes methods based on the same intuitions but carried out within a conventional logic =-=[33]-=-. Our own preliminary experiments with Urban’s methods have been encouraging. 3 The Challenge Our challenge problems are taken from the basic metatheory of System F<:. This system is formed by enrichi... |

2 |
Java light is type-safe|de
- Nipkow, Oheimb
- 1998
(Show Context)
Citation Context ...tness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java =-=[24, 17, 23]-=-. Some other signicant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1] and Typed Assembly Langugages [4]. Inspired by these successes, we seek to make mech... |

2 |
Nominal logic: A order theory of names and binding
- Pitts
- 2001
(Show Context)
Citation Context ... of reasoning about binders based upon a set theory extended with an intrinsic notion of permutation [8]. Pitts followed this up by proposing a new \nominal" logic based upon the idea of permutations =-=[30]-=-. More recent work by Urban proposes methods based on the same intuitions but carried out within a conventional logic [33]. Our own preliminary experiments with Urban's methods have been encouraging. ... |

1 |
Inductive challenge problems, 2000. http://www.cs.nott. ac.uk/ ∼ lad/research/challenges
- Dennis
(Show Context)
Citation Context ...t within the theorem proving community to focus attention on specific areas and to evaluate the relative merits of different tools; these have ranged in scale from benchmark suites and small problems =-=[31,12,5,15,9,21]-=- up to the grand challenges of Floyd, Hoare, and Moore [7,14,20]. We hope that our challenge will have a similarly stimulating effect. Our problems are drawn from the basic metatheory of a call-by-val... |

1 |
Type inference veri Algorithm W in Isabelle/HOL
- Naraschewski, Nipkow
- 1999
(Show Context)
Citation Context ...ions, etc. Large scale formalization of languages is already within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML =-=[6, 22]-=-, semantics for C [25], semantics for Standard ML [32, 34, 13], and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other signicant existing applications of mec... |

1 |
Recursive function de for types with binders
- Norrish
- 2004
(Show Context)
Citation Context ...over untyped lambda-terms [11] and suggest that other inductive structures with binding can be encoded by setting up a correspondence with the untyped lambda terms. Norrish has pursued this direction =-=[26, 27]-=-, but observes that these axioms are cumbersome to use without some assistance from the theorem-proving tool. In particular, the axioms use universal quantication in inductive hypotheses where in inf... |

1 |
Reasoning with the formal de of Standard ML
- Syme
(Show Context)
Citation Context ...ady within reach of current technology. For examples, see the work on proofs of correctness of the DamasMilner type inference algorithm for ML [6, 22], semantics for C [25], semantics for Standard ML =-=[32, 34, 13]-=-, and semantics and proofs of correctness for substantial subsets of Java [24, 17, 23]. Some other signicant existing applications of mechanized metatheory include Foundational Proof Carrying Code [1... |

1 |
A formalised con proof for the -calculus using one-sorted variable names
- Vestergaard, Brotherston
- 2003
(Show Context)
Citation Context ...t meeting the challenge. Asrst-order, named approach very similar insavor to standard informal presentations was used by Vestergaard and Brotherston to formalize some metatheory of untyped -calculus =-=[35, 36]-=-. Their representation requires that each binder initially be assigned a unique name|one aspect of the so-called Barendregt convention. Another popular concrete representation is de Bruijn's nameless ... |