It is difficult to assess hypothetical models in poorly measured domains such as neuroendocrinology. Without a large library of observations to constrain inference, the execution of such incomplete models implies making assumptions. Mutually exclusive assumptions must be kept in separate worlds. We define a general abductive multiple-worlds engine that assesses such models by (i) generating the worlds and (ii) tests if these worlds contain known behaviour. World generation is constrained via the use of relevant envisionment. We describe QCM, a modeling language for compartmental models that can be processed by this inference engine. This tool has been used to nd faults in theories published in international refereed journals; i.e. QCM can detect faults which are invisible to other methods. The generality and computational limits of this approach are discussed. In short, this approach is applicable to any representation that can be compiled into an and-or graph, provided the graphs are not too big or too intricate (fanout<7).

. An abductive framework is described for validating theories using a library of known or desired behaviour. Abduction is known to be NP-hard which suggests that this framework is impractical for anything other than small theories. The computational limits of the framework is therefore explored. We find that abductive validation is a practical tool for the KBS we see in contemporary practice. KEYWORDS: Abduction, validation, computational complexity, expert systems, knowledge acquisition. 1 INTRODUCTION The connection between abduction and other KBS inference tasks (e.g. model-based diagnosis) is well-documented [2, 7]. It would be convenient if we could execute and test our KBS in the same abductive framework. This would remove the need for complicated translations between the executable form of a KBS and its associated test engine. Here we explore KBS validation using HT4, an abductive inference engine. HT4 assumes that the definitive test for a model is that it can reproduce (or c...

Multiple viewpoints are often used in Requirements Engineering to facilitate traceability to stakeholders, to structure the requirements process, and to provide richer modelling by incorporating multiple conflicting descriptions. In the latter case, the need to reason with inconsistent models introduces considerable extra complexity. This paper describes an empirical study of the utility of multiple world reasoning (using abduction) for domain modelling. In the study we used a range of different models (ranging from correct to very incorrect), different fanouts, different amounts of data available from the domain, and different modelling primitives for representing time. In the experiments there was no significant change in the expressive power of models that incorporate multiple conflicting viewpoints. Whilst this does not negate the advantages of viewpoints during requirements elicitation, it does suggest some limits to the utility of viewpoints during requirements modelling. 1. Int...

. This paper suggests that a distinction between knowledge acquisition methods should be made. On the one hand there are methods which aim to help the expert and knowledge engineer analyse what knowledge is involved in solving a particular type of problem and how this problem solving is carried out. These methods are concerned with classifying the different types of problem solving and providing tools and methods to help the knowledge engineer identify the appropriate approach and ensure nothing is omitted.. A different approach to knowledge acquisition focuses on ensuring incremental addition of validated knowledge as mistakes are discovered (validated knowledge here means only that the earlier performance of the system is not degraded by the addition of new knowledge). The organisation of this knowledge is managed by the system rather than the expert and knowledge engineer. This would seem to correspond to human incremental development of expertise. From this perspective...

Situated cognition is not a mere philosophical concern: it has pragmatic implications for current practice in knowledge acquisition. Tools must move from being design-focused to being maintenance-focused. Reuse-based approaches (e.g. using problem solving methods) will fail unless the reused descriptions can be extensively modified to suit the new situation. Knowledge engineers must model not only descriptions of expert knowledge, but also the environment in which a knowledge base will perform. Descriptions of knowledge must be constantly re-evaluated. This re-evaluation process has implications for assessing representations 1.

The work reported in this thesis is motivated by the belief that knowledge-based systems (KBS) research needs to focus more on users ’ needs and cater for the various decision situations in which users will find themselves. To build individual systems that cater for all the activities that may be needed is not feasible or desirable. The problems associated with capturing knowledge are well known and the ability to capture knowledge once and access and manipulate the knowledge in multiple ways is highly desirable. It adds value to the original knowledge and offers all the benefits associated with the reuse of resources. Thus, the problem becomes one of knowledge reuse. The research question pursued in this thesis is “can knowledge captured for one purpose, such as consultation, be reused to support a wide range of alternative purposes, such as critquing or tutoring, allowing the user to answer different types of questions according to their current circumstances”? Further, this question was to be answered in a situated cognition, dynamic knowledge framework. The system developed in this thesis is based on the Multiple Classification Ripple Down Rule (MCRDR) knowledge acquisition and representation technique. MCRDR is a form of casedbased

. Abductive validation for a theory T expressed in language LQCM is known to be exponential. Despite this, abductive validation over LQCM using the HT4 abductive inference engine is a useful technique for a large range of real-world theories. However, doubts persist about LTQCM : a time-based variant of LQCM . In LTQCM , abductive validation is executed for for theories used in long time-based simulations. Here we show that, in the special case where (i) the theory is only measured at a few time intervals and (ii) LTQCM is restricted to L \Gamma TQCM (which contains only bi-state objects connected by symmetrical relations) then temporal abductive validation is practical. 1 Introduction We are used to assessing representations via their soundness, completeness and their tractability. Here we offer an assessment criteria for a language based on its "testability". The assessment criteria proceeds as follows. (i) Define a representation language L. (ii) Define a validation engine for...

Knowledge-based engineering and computational intelligence are expected to become core technologies in the design and manufacturing for the next generation of space exploration missions. The literature is contradictory on how we are to assess such systems. Studies indicate significant disagreement regarding the amount of testing needed for system assessment. The sizes of standard black-box test suites are impractically large since the black-box approach neglects the internal structure of knowledge-based systems. On the contrary, practical results repeatedly indicate that only a few tests are needed to sample the range of behaviors of a knowledge-based program. In this paper, we model testing as a search process over the internal state space of the knowledge-based system. When comparing different test suites, the test suite that examines larger portion of the state space is considered more complete. Our goal is to investigate the trade-off between the completeness criterion and the size of test suites. The results of testing experiment on tens of thousands of mutants of real-world knowledge based systems indicate that a very limited gain in completeness can be achieved through prolonged testing. The use of simple (or random) search strategies for testing appears to be as powerful as testing by more thorough search algorithms.

Models should be able to reproduce the known behaviour of whatever it is they are trying to model. In its most general form, this test is abduction; i.e. the generating an internally-consistent scenario that entails some subset of known observations given certain inputs. Exhaustive abduction (EA) is the generation of all such scenarios. EA can be used to verify a model. If all of the known behaviour cannot be found in any of the generated scenarios, then the model must be faulty. Given that abduction is known to be slow, a reasonable preexperimental intuition is that EA would not be a practical technique for large models. In the study presented here, EAs were executed for a variety of models of different sizes and internal fan-outs. The limits of EA for the current implementation and the studied models implied that EA has some practical utility as a validation tool. Keywords: validation, abduction, hypothesis testing, qualitative reasoning, neuroendocrinology. 1. INTRODUCTION Models...

Testing is a search process and a test suite is complete when the search has examined all the corners of the program. Standard models of test suite sizes are gross over-estimates since they are unaware of the nature of that search space. For example, only a small part of the possible search space is ever exercised in practice. Further, a repeated result is that a few random searches often yields as much information as more thorough search strategies. Hence, only a few tests are needed to sample the range of behaviours of a program.