Objective measurement of test quality is one of the key issues in software testing. It has been a major research focus for the last two decades. Many test criteria have been proposed and studied for this purpose. Various kinds of rationales have been presented in support of one criterion or another. We survey the research work in

Abstract not found

This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

Analysis techniques, such as control-flow, data-flow, and control-dependence, are used for a variety of software engineering tasks, including structural and regression testing, dynamic execution profiling, static and dynamic slicing, and program understanding. To be applicable to programs in languages such as Java and C++, these analysis techniques must account for the effects of exception occurrences and exception-handling constructs; failure to do so can cause the analysis techniques to compute incorrect results and thus, limit the usefulness of the applications that use them. This paper discusses the impact of exception-handling constructs on several analysis techniques. The paper presents techniques to construct representations for programs with exception occurrences and exception-handling constructs, along with algorithms that use these representations to perform the desired analyses. The paper also discusses several software engineering applications that use these analyses. Finally, the paper describes empirical results pertaining to the occurrence of exception-handling constructs in Java programs and their impact on some analysis and software engineering tasks.

Abstract not found

The paper discusses the application of state diagrams in UML to class testing. A set of coverage criteria is proposed based on control and data flow in UML state diagrams and it is shown how to generate test cases satisfying these criteria from UML state diagrams. First, control flow is identified by transforming UML state diagrams into extended finite state machines (EFSMs). The hierarchical and concurrent structure of states is flattened and the broadcast communication is eliminated in the resulting EFSMs. Second, data flow is identified by transforming EFSMs into flow graphs to which conventional data flow analysis techniques can be applied.

Improper management of software evolution, compounded by imprecise, and changing requirements, along with the “short time to market ” requirement, commonly leads to a lack of up-to-date specifications. This can result in software that is characterized by bugs, anomalies and even security threats. Software specification mining is a new technique to address this concern by inferring specifications automatically. In this paper, we propose a novel API specification mining architecture called SMArTIC (Specification Mining Architecture with Trace fIltering and Clustering) to improve the accuracy, robustness and scalability of specification miners. This architecture is constructed based on two hypotheses: (1) Erroneous traces should be pruned from the input traces to a miner, and (2) Clustering related traces will localize inaccuracies and reduce over-generalizationin learning. Correspondingly, SMArTIC comprises four components: an erroneous-trace filtering block, a related-trace clustering block, a learner, and a merger. We show through experiments that the quality of specification mining can be significantly improved using SMArTIC.

In this paper, we model the relation between testing effort, coverage and reliability. We present a logarithmic model that relates testing effort to test coverage (block, branch, c-use or p-use). The model is based on the hypothesis that the enumerables (like branches or blocks) for any coverage measure have different detectability, just like defects have different detectability. This model allows us to relate a test coverage measure directly with defect coverage. Data sets for programs with real defects are used to validate the model. The results are consistent with the known inclusion relationships among block, branch and p-use coverage measures. We show how defect density controls time to next failure. The model can eliminate the variables like test application strategy from consideration. It is suitable for high reliability applications where automatic (or manual) test generation is used to cover enumerables which have not yet been tested.

Abstract In object-oriented testing literature, a class is considered to be a basic unit of testing. A major characteristic of classes is the interaction between data members and member functions. This interaction is represented as definitions and uses of data members in member functions and can be properly modeled with finite state machines (FSM). In this paper, we discuss how FSMs can be effectively used for class testing. We demonstrate how to specify the behavior of classes using FSMs and present a test case generation technique based on FSMs. In our technique, FSMs are transformed into a flow graph from which we can explicitly identify data flows of the FSM. Then we generate test cases using conventional data flow testing techniques upon the flow graph. 1

Software testing and maintenance account for as much as two-thirds of the cost of software production. Program analysis techniques offer the potential to automate testing and maintenance tasks, and thereby reduce the cost of these tasks. An emerging paradigm of software development that promises to enhance software productivity and quality composes software from independently-developed components. The nature of component-based software, however, introduces new problems for applying program analysis techniques to testing and maintenance of such software: the providers of software components develop and test the components independently of the applications that use the components; the users of software components analyze and test their applications without access to source code of the components used in those applications. This paper describes issues and challenges in applying analysis and testing techniques to component-based software and presents an approach to analyzing and testing co...