Evaluation of knowledge acquisition methods remains an important goal; however, evaluation of actual knowledge acquisition is difficult because of the unavailability of experts for adequately controlled studies. This paper proposes the use of simulated experts, i.e., other knowledge based systems as sources of expertise in assessing knowledge acquisition tools. A simulated expert is not as creative or wise as a human expert, but it readily allows for controlled experiments. This method has been used to assess a knowledge acquisition methodology, Ripple Down Rules at various levels of expertise and shows that redundancy is not a major problem with RDR. Introduction Evaluation of knowledge acquisition (KA) methods remains an important goal. Many KA methods have been proposed and many tools have been developed. However, the critical issue for any developer of knowledge based systems (KBS) is to select the best KA technique for the task in hand. This means that papers describing m...

. This paper suggests that a distinction between knowledge acquisition methods should be made. On the one hand there are methods which aim to help the expert and knowledge engineer analyse what knowledge is involved in solving a particular type of problem and how this problem solving is carried out. These methods are concerned with classifying the different types of problem solving and providing tools and methods to help the knowledge engineer identify the appropriate approach and ensure nothing is omitted.. A different approach to knowledge acquisition focuses on ensuring incremental addition of validated knowledge as mistakes are discovered (validated knowledge here means only that the earlier performance of the system is not degraded by the addition of new knowledge). The organisation of this knowledge is managed by the system rather than the expert and knowledge engineer. This would seem to correspond to human incremental development of expertise. From this perspective...

A knowledge acquisition methodology, Ripple Down Rules, has been developed which only allows knowledge to be used in the context in which it is acquired and ensures that only valid rules can be added to a knowledge base. This method has now been used to build a large (2000 rule) medical expert system. This system is in routine use in a Chemical Pathology laboratory providing clinical interpretations for laboratory reports. It has been developed entirely by experts with no knowledge acquisition or programming support or skills. This task was a minor extension to their normal duties. This paper describes the resultant knowledge base and concludes that such knowledge acquisition is very simple. It claims that allowing knowledge to be easily refined is a powerful technique which greatly simplifies dealing with complex domains. INTRODUCTION TO RIPPLE DOWN RULES Ripple down rules (RDR) is a knowledge acquisition methodology and a way of structuring knowledge bases which grew o...