We propose a new framework for black-box conformance testing of real-time systems. The framework is based on the model of partially-observable, non-deterministic timed automata. We argue that partial observability and non-determinism are essential features for ease of modeling, expressiveness and implementability. The framework allows the user to define, through appropriate modeling, assumptions on the environment of the system under test (SUT) as well as on the interface between the tester and the SUT. We consider two types of tests: analog-clock tests and digital-clock tests. Our algorithm to generate analogclock tests is based on an on-the-fly determinization of the specification automaton during the execution of the test, which in turn relies on reachability computations. The latter can sometimes be costly, thus problematic, since the tester must quickly react to the actions of the system under test. Therefore, we provide techniques which allow analog-clock testers to be represented as deterministic timed automata, thus minimizing the reaction time to a simple state jump. We provide algorithms for static or on-the-fly generation of digitalclock tests. These tests measure time only with finite-precision, digital clocks, another essential condition for implementability. We also propose a technique for location, edge and state coverage of the specification, by reducing the problem to covering a symbolic reachability graph. This avoids having to generate too many tests. We report on a prototype tool TTG and two case studies: a lighting device and the Bounded Retransmission Protocol. Experimental results obtained by applying TTG on the Bounded Retransmission Protocol show that only a few tests suffice to cover thousands of reachable symbolic states in the specification.

This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

We present an implemented technique for generating test cases from state machine specifications. The work is motivated by a need for testing of protocols and services developed by the company Mobile Arts. We have developed a syntax for description of state machines extended with data variables. From such state machines, test cases are generated by symbolic execution. The test cases are symbolically represented; concrete test cases are generated by instantiation of data parameters.

Abstract. We propose a new framework for black-box conformance testing of real-time systems, based on the model of timed automata. The framework is expressive: it can fully handle partially-observable, nondeterministic timed automata. It also allows the user to define, through appropriate modeling, assumptions on the environment of the system under test (SUT) as well as on the interface between the tester and the SUT. The framework is implementable: tests can be implemented as finite-state machines accessing a finite-precision digital clock. We propose, for this framework, a set of test-generation algorithms with respect to different coverage criteria. We have implemented these algorithms in a prototype tool called TTG. Experimental results obtained by applying TTG on the Bounded Retransmission Protocol show that only a few tests suffice to cover thousands of reachable symbolic states in the specification. 1

Introduction. The goal of testing is to gain confidence in a physical computer based system by means of executing it. More than one third of typical project resources is spent on testing and still it remains ad-hoc, based on heuristics, and error-prone. Moreover, it is estimated that 99 % of processors produced today are targeted for embedded applications. Real-time and embedded systems require a special attention to timing where the moment of input and output event appearance is as important

This paper presents a game-theoretic approach to the testing of uncontrollable real-time systems. By modelling the systems with Timed I/O Game Automata and specifying the test purposes as Timed CTL formulas, we employ a recently developed timed game solver UPPAAL-TIGA to synthesize winning strategies, and then use these strategies to conduct conformance testing of the systems. The testing process is proved to be sound and complete with respect to the given test purposes. Case study and preliminary experimental results indicate that this is a viable approach to real-time system testing.

In this paper, we describe how the real-time verification tool UPPAAL has been extended to support automatic generation of time-optimal test suites for conformance testing. Such test suites are derived from a network of timed automata specifying the expected behaviour of the system under test and its environment. To select test cases, we use coverage criteria specifying structural criteria to be fulfilled by the test suite. The result is optimal in the sense that the set of test cases in the test suite requires the shortest possible accumulated time to cover the given coverage criterion. The main contributions of this paper are: (i) a modified reachability analysis algorithm in which the coverage of given criteria is calculated in an on-the-fly manner, (ii) a technique for efficiently manipulating the sets of covered elements that arise during the analysis, and (iii) an extension to the requirement specification language used in UP-PAAL, making it possible to express a variety of coverage criteria. 1.

Testing concurrent software is difficult due to problems with inherent non-determinism. In previous work, we have presented a method and tool support for the testing of concurrent Java components. In this paper, we extend that work by presenting and discussing techniques for testing Java thread interrupts and timed waits. Testing thread interrupts is important because every Java component that calls wait must have code dealing with these interrupts. For a component that uses interrupts and timed waits to provide its basic functionality, the ability to test these features is clearly even more important. We discuss the application of the techniques and tool support to one such component, which is a non-trivial implemenation of the readerswriters problem. 1

COVER is new test-case generation tool for timed systems. It generates test cases from a timed automata model of a system to be tested, and a coverage criteria expressed in an observer language. In this paper, we describe the current architecture of the tool, its input languages, and a case study in which tool has been applied in an industrial setting to test that a WAP gateway conform to its specification.

Abstract not found