Abstract not found

Abstract. Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, the Extreme Programming approach relies on frequent unit testing. In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language’s runtime assertion checker to decide whether methods are working correctly, thus automating the writing of unit test oracles. These oracles can be easily combined with hand-written test data. Instead of writing testing code, the programmer writes formal specifications (e.g., pre- and postconditions). This makes the programmer’s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable

We present TestEra, a novel framework for automated testing of Java programs. TestEra automatically generates all non-isomorphic test cases, within a given input size, and evaluates correctness criteria. As an enabling technology, TestEra uses Alloy, a first-order relational language, and the Alloy Analyzer. Checking a program with TestEra involves modeling the correctness criteria for the program in Alloy and specifying abstraction and concretization translations between instances of Alloy models and Java data structures. TestEra produces concrete Java inputs as counterexamples to violated correctness criteria. This paper discusses TestEra's analyses of several case studies: methods that manipulate singly linked lists and red-black trees, a naming architecture, and a part of the Alloy Analyzer.

This paper presents the operational difference technique for generating, augmenting, and minimizing test suites. The technique is analogous to structural code coverage techniques, but it operates in the semantic domain of program properties rather than the syntactic domain of program text. The operational difference technique automatically selects test cases; it assumes only the existence of a source of test cases. The technique dynamically generates operational abstractions (which describe observed behavior and are syntactically identical to formal specifications) from test suite executions. Test suites can be generated by adding cases until the operational abstraction stops changing. The resulting test suites are as small, and detect as many faults, as suites with 100% branch coverage, and are better at detecting certain common faults.

This paper presents general criteria for generating test inputs from state-based specifications. Software testing can only be formalized and quantified whena solid basis for test generation can be defined. Formal specifications of complex systems represent a significant opportunity for testing because they precisely describe what functions the software is supposed to provide in a form that can easily be manipulated. These techniques provide coverage criteria that are based on the specifications, and are made up of several parts, including test prefixes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming specifications to tests. Empirical results from a comparative case study application of these criteria are presented.

Although the majority of software testing in industry is conducted at the system level, most formal research has focused on the unit level. As a result, most system level testing techniques are only described informally. This paper presents formal testing criteria for system level testing that are based on formal specifications of the software. Software testing can only be formalized and quantified when a solid basis for test generation can be defined. Formal specifications represent a significant opportunity for testing because they precisely describe what functions the software is supposed to provide in a form that can be automatically manipulated. This paper presents general criteria for generating test inputs from state-based specifications. The criteria include techniques for generating tests at several levels of abstraction for specifications (transition predicates, transitions, pairs of transitions and sequences of transitions). These techniques provide coverage criteria that are based on the specifications, and are made up of several parts, including test prefixes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming specifications to tests. These criteria have been applied to a case study to compare their ability to detect seeded faults.

This paper argues that with the advent of explicitly speci ed software architectures, testing can be done e ectively at the architectural level. A software architecture speci-cation provides a solid foundation for developing a plan for testing at this level. We propose several architecture-based test criteria based on the Chemical Abstract Machine model of software architecture. An architectural (integration) test plan, developed by applying selected of these criteria, can be used to assess the architecture itself or to test the implementation's conformance with the architecture. This facilitates detecting defects earlier in the software lifecycle, enables leveraging software testing costs across multiple systems developed from the same architecture, and also leverages the e ort put into developing a software architecture. 1

This report presents a new, automatic technique to assess whether replacing a component of a software system by a purportedly compatible component may change the behavior of the system. The technique operates before integrating the new component into the system or running system tests, permitting quicker and cheaper identification of problems. It takes into account the system’s use of the component, because a particular component upgrade may be desirable in one context but undesirable in another. No formal specifications are required, permitting detection of problems due either to errors in the component or to errors in the system. Both external and internal behaviors can be compared, enabling detection of problems that are not immediately reflected in the output. The technique generates an operational abstraction for the old component in the context of the system, and one for the new component in the context of its test suite. An operational abstraction is a set of program properties that generalizes over observed run-time behavior. Modeling a system as divided into modules, and taking into account the control and data flow between the modules, we formulate a logical condition to guarantee that the system’s behavior is preserved across a component replacement. If automated logical comparison indicates that the new component does not make all the guarantees that the old one did, then

Effective testing of critical systems has been hampered by the lack of a cost-effective method for deciding the correctness of a program's behavior under test. Using formal specifications to describe the critical system properties and then checking test results against these specifications overcomes these problems. If these test oracles, which are mechanisms for determining whether a test passes or fails, are efficient, they can be combined with automatic test generation to cost-effectively automate the testing of large numbers of testcases that more adequately cover the system requirements and structure. This paper presents a algorithm for automatically deriving efficient test oracles from Graphical Interval Logic (GIL) [5], which is a graphical temporal logic that is easier for non-experts to understand than many formal languages. To develop efficient test oracles from GIL, we convert the specifications into automata that can be checked in time linear in the length of the trace. Addi...

Software testing consumes a large percentage of total software development costs. Yet, it is still usually performed manually in a non rigorous fashion. While techniques, and limited automatic support, for the generation of test data from the actual code of the system under test have been well researched, test cases generation from a high level specification of the intended behaviour of the system being developed has hardly been addressed. In this thesis we present a rationale for using tests derived from high level formal specifications and then set to find an efficient technique for the generation of adequate test sets from specifications written in our study language, VDM-SL. In this work, we formalise the traditional high level partitioning technique used in a previously researched test cases generator prototype, and extend it to take the semantics of VDM-SL fully into account. We then discuss, and illustrate, the shortcomings of the technique as used, which results in too few test...