We present and evaluate an automatic tool for extracting algebraic specifications from Java classes. Our tool maps a Java class to an algebraic signature and then uses the signature to generate a large number of terms. The tool evaluates these terms and based on the results of the evaluation, it proposes equations. Finally, the tool generalizes equations to axioms and eliminates many redundant axioms. Since our tool uses dynamic information, it is not guaranteed to be sound or complete. However, we manually inspected the axioms generated in our experiments and found them all to be correct.

Object-oriented programming consists of several different levels of abstraction, namely the algorithmic level, class level, cluster level, and system level. The testing of object-oriented software at the algorithmic and system levels is similar to conventional programming testing. Testing at the class and cluster levels poses new challenges. Since methods and objects may interact with one another dynamically with unforeseen combinations and invocations, they are much more complex to simulate and test than the static hierarchy of functional calls in conventional programs. In this paper, we propose a methodology for object-oriented software testing at the class and cluster levels. In class-level testing, it is essential to determine whether objects produced from the execution of implemented systems would preserve the properties defined by the specification, such as behavioral equivalence and non-equivalence. Our class-level testing methodology addresses both of these aspects. For the testing of behavioral equivalence, we propose to select fundamental pairs of equivalent ground terms as test cases using a black-box technique based on algebraic specifications, and then determine by means of a white-box technique whether the objects resulting from executing such test cases are observationally equivalent. To address the testing of behavioral non-equivalence, we have identified and analyzed several non-trivial problems in the current literature. We propose to classify term equivalence into four types, thereby setting up new concepts and deriving important properties. Based on these results, we propose an approach to deal with the problems in the generation of non-equivalent ground terms as test cases. Relatively little research has contributed to cluster-level testing. In this paper, we also discuss black-box testing at the cluster level. We illustrate the feasibility of using Contract, a formal specification language for the behavioral dependencies and interactions among cooperating objects of different classes in a given cluster. We propose an approach to test the interactions among different classes using every individual message-passing rule in the given Contract specification. We also present an approach to examine the interactions among composite message-passing rules. We have developed four testing tools to support our methodology.

Testing is a trade-off between increased confidence in the correctness of the implementation under test and constraints on the amount of time and effort that can be spent in testing. Therefore, the coverage, or adequacy of the test suite, becomes a very important issue. In this paper, we analyze basic ideas underlying the techniques for fault coverage analysis and assurance mainly developed in the context of protocol conformance testing based on finite state models. Special attention is paid to parameters which determine the testability of a given specification and influence the length of a test suite which guarantees complete fault coverage. We also point out certain issues which need further study. 1. INTRODUCTION Testing is a critical phase in the development life cycle for any hardware and software system, and in particular for communication systems for which it ensures the conformance to the relevant standards and compatibility or interoperability with other systems. Testing con...

Because of the growing importance of object-oriented programming, a number of testing strategies have been proposed. They are based either on pure black-box or white-box techniques. We propose in this paper a methodology to integrate the black- and white-box techniques. The black-box technique is used to select test cases. The white-box technique is mainly applied to decide whether two objects resulting from the program execution of a test case are observationally equivalent. It is also used to select test cases in some situations. We define the concept of a fundamental pair as a pair of equivalent terms that are formed by replacing all the variables on both sides of an axiom by normal forms. We prove that an implementation is consistent with respect to all equivalent terms if and only if it is consistent with respect to all fundamental pairs. In other words, the testing coverage of fundamental pairs is as good as that of all possible term rewritings, and hence we need only concentrate on the testing of fundamental pairs. Our strategy is based on mathematical theorems. According to the strategy, we propose an algorithm for selecting a finite set of fundamental pairs as test cases. Given a pair of equivalent terms as a test case, we should then determine whether the objects that result from executing the implemented program are observationally equivalent. We prove, however, that the observational equivalence of objects cannot be determined using a finite set of observable contexts (namely, operation sequences ending with an observer function) derived from any black-box technique. Hence we supplement our approach with a "relevant observable context" technique, which is a white-box technique, to determine the observational equivalence. The relevant observable contexts are constructed from a Data Member Relevance Graph, which is an abstraction of the given implementation for a given specification. A semi-automatic tool has been developed to support this technique.

Abstract. We present the PathCrawler prototype tool for the automatic generation of test-cases satisfying the rigorous all-paths criterion, with a user-defined limit on the number of loop iterations in the covered paths. The prototype treats C code and we illustrate the test-case generation process on a representative example of a C function containing data-structures of variable dimensions, loops with variable numbers of iterations and many infeasible paths. PathCrawler is based on a novel combination of code instrumentation and constraint solving which makes it both efficient and open to extension. It suffers neither from the approximations and complexity of static analysis, nor from the number of executions demanded by the use of heuristic algorithms in function minimisation and the possibility that they fail to find a solution. We believe that it demonstrates the feasibility of rigorous and systematic testing of sequential programs coded in imperative languages. 1

Context-sensitive middleware-based software is an emerging kind of ubiquitous computing application. The components of such software communicate proactively among themselves according to the situational attributes of their environments, known as the ``contexts''. The actual process of accessing and updating the inter-component contexts lies with the middleware. The latter invokes the relevant local and remote operations whenever any context inscribed in the situation-aware interface is satisfied. Since the applications operate in a highly dynamic environment, the testing of context-sensitive software is complex and challenging. Metamorphic testing is a property-based testing strategy. It recommends that, even if a test case does not reveal any failure, follow-up test cases should be further constructed from the original to check whether the software satisfies some necessary conditions of the problem to be implemented. This paper proposes to use isotropic properties of contexts as metamorphic relations for testing context-sensitive software. For instance, distinct points on the same isotropic curve of contexts would entail comparable responses by the components. This notion of testing context relations is novel, robust, and intuitive to users.

Web Services are the basic building blocks of next generation Internet applications, based on dynamic service discovery and composition.

This paper describes the specification-based testing and analysis tools, and associated processes, that were used to develop and certify safety-critical avionics systems in an industrial organization. These tools comprise an integrated development environment supporting specification acquisition and analysis, requirement-based automatic test vector generation, test coverage analysis, test driver generation, and test results analysis. The paper describes the specification model, method, development environment, and tool qualification approach. The capabilities of the automatic test generator are compared with foundational concepts and related testing strategies and mechanisms. 1.

In partition analysis we divide the input domain to form subdomains on which the system’s behaviour should be uniform. Boundary value analysis produces test inputs near each subdomain’s boundaries to find failures caused by the boundaries being incorrectly implemented. However, boundary value analysis can be adversely affected by coincidental correctness — the system produces the expected output for the wrong reason. This paper shows how boundary value analysis can be adapted in order to reduce the opportunity for coincidental correctness. The main contribution is to automated test data generation in which one cannot rely on the expertise of a tester.

We present a method for the automatic generation of tests satisfying the all-paths criterion, with a user-defined limit, k, on the number of loop iterations in the covered paths. We have implemented a prototype for C code. We illustrate our approach on a representative example of a C function containing data-structures of variable dimensions, loops with variable numbers of iterations and many infeasible paths. We explain why our method is efficient enough to scale up to the unit testing of realistic programs. It is flexible enough to take into account certain specifications of the tested code. This is why we believe that it could become the cornerstone of a fully automatic grey-box testing process, based on the novel combination of code instrumentation and constraint solving described here. 1.