We report on a study to determine the maturity of di#erent verification and validation technologies (V&V) on a representative example of NASA flight software. The study consisted of a controlled experiment where three technologies (static analysis, runtime analysis and model checking) were compared to traditional testing with respect to their ability to find seeded errors in a prototype Mars Rover. What makes this study unique is that it is the first (to the best of our knowledge) to do a controlled experiment to compare formal methods based tools to testing on a realistic industrial-size example where the emphasis was on collecting as much data on the performance of the tools and the participants as possible. The paper includes a description of the Rover code that was analyzed, the tools used as well as a detailed description of the experimental setup and the results. Due to the complexity of setting up the experiment, our results can not be generalized, but we believe it can still serve as a valuable point of reference for future studies of this kind. It did confirm the belief we had that advanced tools can outperform testing when trying to locate concurrency errors. Furthermore the results of the experiment inspired a novel framework for testing the next generation of the Rover.

This paper presents a mechanizable framework for software developmentby refinement. The framework is based on a category of higher-order specifications. The key idea is representing knowledge about programming concepts, such as algorithm design, datatype refinement, and expression simplification, by means of taxonomies of specifications and morphisms.

Software is increasingly complex and is used in increasingly critical applications. Sophisticated techniques are available for verifying that software systems work correctly, but these techniques can be very difficult and expensive to use. Researchers have developed tools to automatically verify software models, but using these tools can still be very costly, in terms of manual effort and expertise required to build accurate models and to formally specify required properties, and also in terms of the time and memory required to run these tools. Much work has been done to simplify the process of building software models and to improve the performance of verification tools, resulting in a variety of different modeling languages, each with features designed to reduce effort or improve performance for certain types of input models, and a range of verification tools, each with a different set of strategies available for reducing time and memory requirements. It can be difficult to determine which verification strategy is best for a particular software system. Others have observed complementary relationships between tools and have argued that there is no single best tool—that as users’ needs change the choice of tool should change as well. This dissertation provides further evidence for complementary relationships between verification tools, specifically considering tools available for specifications of synchronous software systems

This paper motivates and describes a new paradigm and discipline for knowledge, software, and system development and maintenance. This paradigm promises to improve system quality and make systems development and maintenance faster and cheaper.

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington