Results 1  10
of
15
Random testing in isabelle/hol
 Software Engineering and Formal Methods (SEFM 2004
, 2004
"... When developing nontrivial formalizations in a theorem prover, a considerable amount of time is devoted to “debugging ” specifications and conjectures by failed proof attempts. To detect such problems early in the proof and save development time, we have extended the Isabelle theorem prover with a ..."
Abstract

Cited by 42 (2 self)
 Add to MetaCart
When developing nontrivial formalizations in a theorem prover, a considerable amount of time is devoted to “debugging ” specifications and conjectures by failed proof attempts. To detect such problems early in the proof and save development time, we have extended the Isabelle theorem prover with a tool for testing specifications by evaluating propositions under an assignment of random values to free variables. Distribution of the test data is optimized via mutation testing. The technical contributions are an extension of earlier work with inductive definitions and a generic method for randomly generating elements of recursive datatypes. 1.
Functional Testing in the Focal environment
"... Abstract. This article presents the generation and test case execution under the framework Focal. In the programming language Focal, all properties of the program are written within the source code. These properties are considered, here, as the program specification. We are interested in testing the ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
Abstract. This article presents the generation and test case execution under the framework Focal. In the programming language Focal, all properties of the program are written within the source code. These properties are considered, here, as the program specification. We are interested in testing the code against these properties. Testing a property is split in two stages. First, the property is cut out in several elementary properties. An elementary property is a tuple composed of some preconditions and a conclusion. Lastly, each elementary property is tested separately. The preconditions are used to generate and select the test cases randomly. The conclusion allows us to compute the verdict. All the testing process is done automatically. 1
P.: Testing Data Types Implementations from Algebraic Specifications
, 2008
"... Abstract. Algebraic specifications of data types provide a natural basis for testing data types implementations. In this framework, the conformance relation is based on the satisfaction of axioms. This makes it possible to formally state the fundamental concepts of testing: exhaustive test set, test ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
Abstract. Algebraic specifications of data types provide a natural basis for testing data types implementations. In this framework, the conformance relation is based on the satisfaction of axioms. This makes it possible to formally state the fundamental concepts of testing: exhaustive test set, testability hypotheses, oracle. Various criteria for selecting finite test sets have been proposed. They depend on the form of the axioms, and on the possibilities of observation of the implementation under test. This last point is related to the wellknown oracle problem. As the main interest of algebraic specifications is data type abstraction, testing a concrete implementation raises the issue of the gap between the abstract description and the concrete representation. The observational semantics of algebraic specifications bring solutions on the basis of the socalled observable contexts. After a description of testing methods based on algebraic specifications, the chapter gives a brief presentation of some tools and case studies, and presents some applications to other formal methods involving datatypes.
Random testing in PVS
 In: Workshop on Automated Formal Methods (AFM
, 2006
"... Abstract. Formulas are difficult to formulate and to prove, and are often invalid during specification development. Testing formulas prior to attempting any proofs could potentially save a lot of effort. Here we describe an implementation of random testing in the PVS verification system. 1 ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
Abstract. Formulas are difficult to formulate and to prove, and are often invalid during specification development. Testing formulas prior to attempting any proofs could potentially save a lot of effort. Here we describe an implementation of random testing in the PVS verification system. 1
Verifying haskell programs by combining testing and proving
 In Proceedings of the Third International Conference on Quality Software
"... We propose a method for improving confidence in the correctness of Haskell programs by combining testing and proving. Testing is used for debugging programs and specification before a costly proof attempt. During a proof development, testing also quickly eliminates wrong conjectures. Proving helps u ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We propose a method for improving confidence in the correctness of Haskell programs by combining testing and proving. Testing is used for debugging programs and specification before a costly proof attempt. During a proof development, testing also quickly eliminates wrong conjectures. Proving helps us to decompose a testing task in a way that is guaranteed to be correct. To demonstrate the method we have extended the Agda/Alfa proof assistant for dependent type theory with a tool for random testing. As an example we show how the correctness of a BDDalgorithm written in Haskell is verified by testing properties of component functions. We also discuss faithful translations from Haskell to type theory.
Automatic Proof and Disproof in Isabelle/HOL
, 2011
"... Isabelle/HOL is a popular interactive theorem prover based on higherorder logic. It owes its success to its ease of use and powerful automation. Much of the automation is performed by external tools: The metaprover Sledgehammer relies on resolution provers and SMT solvers for its proof search, the c ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Isabelle/HOL is a popular interactive theorem prover based on higherorder logic. It owes its success to its ease of use and powerful automation. Much of the automation is performed by external tools: The metaprover Sledgehammer relies on resolution provers and SMT solvers for its proof search, the counterexample generator Quickcheck uses the ML compiler as a fast evaluator for ground formulas, and its rival Nitpick is based on the model finder Kodkod, which performs a reduction to SAT. Together with the Isar structured proof format and a new asynchronous user interface, these tools have radically transformed the Isabelle user experience. This paper provides an overview of the main automatic proof and disproof tools.
Testing Noninterference, Quickly
"... Informationflow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on doomed proofs for broken definitions, we advocate modern random testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a pr ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Informationflow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on doomed proofs for broken definitions, we advocate modern random testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a propertybased randomtesting tool, to guide the design of a simple informationflow abstract machine. We find that both sophisticated strategies for generating welldistributed random programs and readily falsifiable formulations of noninterference properties are critically important. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for all these bugs.
Constraint Reasoning in FocalTest
"... Abstract. Program testing implies selecting test data from the program input space. In many cases, test data satisfying userspecified properties or preconditions (a.k.a. positive test data) are required. However, current automatic test data generation techniques adopt direct generateandtest approa ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Abstract. Program testing implies selecting test data from the program input space. In many cases, test data satisfying userspecified properties or preconditions (a.k.a. positive test data) are required. However, current automatic test data generation techniques adopt direct generateandtest approaches for this task. In FocalTest, the testing tool of the Focal correctbyconstruction environment for developing certified objectoriented functional programs, test data are generated at random and rejected when they do not satisfy preconditions. In this paper, we improve FocalTest with a testandgenerate approach, through the usage of constraint reasoning. A particular difficulty is the handling of function calls in the preconditions as they require constraint reasoning on conditionals and pattern matching which introduce disjunctions in constraint systems. Our experimental results show that a nonnaive implementation of constraint reasoning on these constructions outperform traditional generation techniques when used to find test data for testing properties. 1
Formalising finfuns — generating code for functions as data from Isabelle/HOL
 of Lecture Notes in Computer Science
, 2009
"... Abstract. FinFuns are total functions that are constant except for a finite set of points, i.e. a generalisation of finite maps. We formalise them in Isabelle/HOL and present how to safely set up Isabelle’s code generator such that operations like equality testing and quantification on FinFuns be ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract. FinFuns are total functions that are constant except for a finite set of points, i.e. a generalisation of finite maps. We formalise them in Isabelle/HOL and present how to safely set up Isabelle’s code generator such that operations like equality testing and quantification on FinFuns become executable. On the code output level, FinFuns are explicitly represented by constant functions and pointwise updates, similarly to associative lists. Inside the logic, they behave like ordinary functions with extensionality. Via the update/constant pattern, a recursion combinator and an induction rule for FinFuns allow for defining and reasoning about operators on FinFuns that directly become executable. We apply the approach to an executable formalisation of sets and use it for the semantics for a subset of concurrent Java. 1
The New Quickcheck for Isabelle Random, Exhaustive and Symbolic Testing Living Under One Roof
"... Abstract. The new Quickcheck is a counterexample generator for Isabelle/HOL that uncovers faulty specifications and invalid conjectures using various testing strategies. The previous Quickcheck only tested conjectures by random testing. The new Quickcheck extends the previous one and integrates two ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. The new Quickcheck is a counterexample generator for Isabelle/HOL that uncovers faulty specifications and invalid conjectures using various testing strategies. The previous Quickcheck only tested conjectures by random testing. The new Quickcheck extends the previous one and integrates two novel testing strategies: exhaustive testing with concrete values; and symbolic testing, evaluating conjectures with a narrowing strategy. Orthogonally to the strategies, we address two general issues: First, we extend the class of executable conjectures and specifications, and second, we present techniques to deal with conditional conjectures, i.e., conjectures with restrictive premises. We evaluate the testing strategies and techniques on a number of specifications, functional data structures and a hotel key card system. 1