Much of the work on validation and verification of knowledge based systems (KBSs) has been done in terms of implementation languages (mostly rule-based languages). Recent papers have argued that it is advantageous to do validation and verification in terms of a more abstract and formal specification of the system. However, constructing such formal specifications is a difficult task. This paper proposes the use of formal specification languages for KBS-development that are closely based on the structure of informal knowledge-models. The use of such formal languages has as advantages that (i) we can give strong support for the construction of a formal specification, namely on the basis of the informal description of the system; and (ii) we can use the structural correspondence to verify that the formal specification does indeed capture the informally stated requirements. This paper has been submitted to the Journal of Human Computer Studies (formerly the Journal of Man Machine Studies)....

: We argue that adding a requirement of evaluation and testing fundamentally changes KBS practice. In particular: (i) a fundamental change to the symbol-level representation in KBS; (ii) a rejection of certain unnecessary knowledge-level distinctions; (iii) a fundamental change to the inference engine of KBS; and (iv) a basic computational limit to the size and internal complexity of the models we create via knowledge acquisition. 1. INTRODUCTION It would be convenient if KBS evaluation was neutral with respect to KBS practice. If an evaluation module was merely a post-hoc bolt-on, then its design could be deferred until after a system was developed. However, if evaluation adds extra requirements and restrictions to the KBS process, then the design of an evaluation module must be integrated with the system it will test. This paper argues for the inconvenient latter position. Models constructed in vague domains (defined below) are possibly inaccurate. Possibly inaccurate models must b...

The work reported in this thesis is motivated by the belief that knowledge-based systems (KBS) research needs to focus more on users ’ needs and cater for the various decision situations in which users will find themselves. To build individual systems that cater for all the activities that may be needed is not feasible or desirable. The problems associated with capturing knowledge are well known and the ability to capture knowledge once and access and manipulate the knowledge in multiple ways is highly desirable. It adds value to the original knowledge and offers all the benefits associated with the reuse of resources. Thus, the problem becomes one of knowledge reuse. The research question pursued in this thesis is “can knowledge captured for one purpose, such as consultation, be reused to support a wide range of alternative purposes, such as critquing or tutoring, allowing the user to answer different types of questions according to their current circumstances”? Further, this question was to be answered in a situated cognition, dynamic knowledge framework. The system developed in this thesis is based on the Multiple Classification Ripple Down Rule (MCRDR) knowledge acquisition and representation technique. MCRDR is a form of casedbased

Knowledge-based engineering and computational intelligence are expected to become core technologies in the design and manufacturing for the next generation of space exploration missions. The literature is contradictory on how we are to assess such systems. Studies indicate significant disagreement regarding the amount of testing needed for system assessment. The sizes of standard black-box test suites are impractically large since the black-box approach neglects the internal structure of knowledge-based systems. On the contrary, practical results repeatedly indicate that only a few tests are needed to sample the range of behaviors of a knowledge-based program. In this paper, we model testing as a search process over the internal state space of the knowledge-based system. When comparing different test suites, the test suite that examines larger portion of the state space is considered more complete. Our goal is to investigate the trade-off between the completeness criterion and the size of test suites. The results of testing experiment on tens of thousands of mutants of real-world knowledge based systems indicate that a very limited gain in completeness can be achieved through prolonged testing. The use of simple (or random) search strategies for testing appears to be as powerful as testing by more thorough search algorithms.

We develop a general abductive description of testing models. We find that this testing process is fundamentally slow and cannot be conducted exhaustively. Consequently, we argue that the usual case for model testing is nonexhaustive testing; i.e. some subset of the possible tests are chosen and executed. Note that if the tests result in model refinement, then different tests can result in different models. This leads to the hypothesis that different individuals form different "opinions" (i.e. models) about the world as a result of the different examples they push through their models. We prefer this symbolic explanation for situated semantics to non-symbolic proposals (e.g. neural).

Knowledge refinement tools have commonly been applied to diagnostic applications. This paper considers the refinement of a design application. It explores the differences in knowledge content and problem-solving steps for design rather than diagnosis systems, and investigates additional refinement operators. Although the problem-solving task itself tends to be more complex, this in fact puts more constraints on the results of the problem-solving and so the evidence on which the refinement is based can be more rich. Results are provided for a real tablet formulation system, TFS, and experience is reported for two types of refinement tasks: debugging to correct faulty problem-solving of an early version of TFS; maintenance of TFS when the formulation task is altered by a change in company policy. 1. Introduction Knowledge refinement is the process of changing knowledge in a knowledge-based system (KBS) in reaction to evidence that the KBS is not producing correct solutions. The KBS is ...

This paper examines how formal specification techniques can support the verification and validation (V&V) of knowledge-based systems. Formal specification techniques provide levels of description which support both verification and validation, and V&V techniques feed back to assist the development of the specifications. Developing

. A new framework is proposed in which software engineering (SE) is the construction of a search space and knowledge engineering (KE) is the constructing the intelligence to control the traversal of that space. Conventional information systems and object-oriented notations can specify the search space. An abductive inference engine can 5 implement the intelligent control. This unified framework supports conventional SE/KE approaches, plus automatic screen generation, conflict resolution in requirements capture, code library management, optimised code generation, and automatic testing tools. 5000 words. Submitted to the 10th European Workshop on Knowledge Ac- 10 quisition, Modeling, and Management Sant Feliu de Guixols, Catalonia, Spain October 15 - 18, 1997 1 Introduction Conventional software descriptions can be divided into several perspectives that include the events and activities in which data is processed (x2.2). Knowledge- 15 level modeling [20, 21] adds an extra perspective...

Ontologies have become popular in the Articial Intelligence community as a way to standardise representation of domain knowledge. Despite their advocated use in areas such as knowledge sharing and reuse there is little discussion in the community regarding their application in other areas. In this paper we explore the use of ontologies in improving systems engineering reliability by consistency checking with respect to ontological axioms - making it possible to reason about the correctness of an application with respect to ontological constraints. We applied this approach to diverse areas, each of which demonstrates a dierent use: deploying ontological axioms in business process modelling and enriching the axiomatisation of an Air Campaign Planning(ACP)-based application; identication of ontological constraints in ecological modelling; and evaluation of ontologies on system dynamics theory at the application level. 1.

Abstract. As their field of application has evolved and matured, the importance of verifying knowledge-based systems is now widely recognized. Nevertheless, some problems have remained. In this paper, we address the poor scalability to larger systems of the computation methods commonly applied to rule-chain anomaly checking. To tackle this problem, we introduce a novel anomaly checking method based on binary decision diagrams (BDDs), a technique emanating mainly from the hardware design community. In addition, we present empirical evidence of its computational efficiency, especially on rule bases with a deeper inference space. 1