Almost all machine learning algorithms---be they for regression, classification or density estimation---seek hypotheses that optimize a score on training data. In most interesting cases, however, full global optimization is not feasible and local search techniques are used to discover reasonable solutions. Unfortunately,

We describe ARACHNE, a concept formation system that, uses explicit constraints on tree structure and local restructuring operators to produce well-formed probabilistic concept trees. We also present a quantitative measure of tree quality and compare the system's performance in artificial and natural domains to that of COBWEB, a well-known concept formation algorithm. The results suggest that ARACHNE frequently constructs higher-quality trees than COBWEB, while still retaining the ability to make accurate predictions. 1

Although many algorithms for learning from examples have been developed and many comparisons have been reported, there is no generally accepted benchmark for classifier learning. The existence of a standard benchmark would greatly assist such comparisons. Sixteen dimensions are proposed to describe classification tasks. Based on these, thirteen real-world and synthetic datasets are chosen by a set covering method from the UCI Repository of machine learning databases to form such a benchmark.

A machine learning (ML) system which combines lattice-based and instance-based learning (IBL) techniques, is motivated and developed in this paper. We describe an IBL system over lattice theory called IGLUE that significantly improved both the complexity and accuracy of lattice-based learning systems. For this purpose, IGLUE uses the entropy function to select relevant lattice nodes, then extracts a set of new numerical features from the original set of boolean features, and finally applies a nearest neighbor technique with the Mahanalobis distance as the similarity measure between redescribed instances. IGLUE treats only domains described with symbolic features. In this paper, we present results of experiments we carried out to assess how well IGLUE performs on real problems, with other similarity measures and selection functions. We combine three selection functions and three similarity measures, and thus run nine experiments. We compare the performance of these combined st...

We consider the problem of learning conditional independencies, expressed as a Markov network, from a probabilistic model. An efficient algorithm employing a greedy search has been developed earlier with promising empirical results. However, two issues were not addressed. First, the reason why the myopic search works so well globally has not been fully understood. Second, whether the algorithm can find a correct Markov network in all cases has not been formally established. In this paper, we prove that, for any given probabilistic model, the algorithm will always produce a Markov network whose structure is an independence map of the underlying model and whose associated probability distribution is identical to the underlying model. The proof also offers deeper insight into the algorithm's working mechanism. As the problem of learning a minimal independence map of a given probabilistic model is NP-hard in general, our polynomial time algorithm does not guarantee minimality in all cases....

This thesis describes the architecture of learning systems which can explain their decisions through a rule-based knowledge representation. Two problems in learning are addressed: pattern classification and function approximation. In Part I, a pattern classifier for discrete-valued problems is presented. The system utilizes an information-theoretic algorithm for constructing informative rules from example data. These rules are then used to construct a computational network to perform parallel inference and posterior probability estimation. The network can be extended incrementally; that is, new data can be incorporated without repeating the training on previous data. It is shown that this technique performs comparably with other techniques including the backpropagation network while having unique advantages in incremental learning capability, training efficiency, and knowledge representation. Examples are shown of rulebased classification and explanation. In Part II, we present a metho...

this paper, we assume data are represented by a relational database in which information about individual objects in a domain is represented by a set of tuples of attribute values. Adopting the view of `learning by examples' from Artificial Intelligence (AI), we may regard a database as a set of training examples. The objective of one form of learning is to produce a classification rule in a disjunctive normal form (DNF) for a particular concept or class. A learned rule can be generated using the vocabulary of attributes. We shall call the set of attributes used in the database a basis set (Genesereth 1987), and call the learning process induction using attributes. For complex domains, the rule generated using the basis set may contain too many conjuncts. One way to make the rule more compact, as well as more general, is to partition attribute values for each attribute using a concept hierarchy. The rule is then represented in a higher level language than the original basis set. We shall call this further generalization

MacLib is a Computerized Reference System to assist pathologists in the differentiation of brain tumors. The system provides a consulting module with collection of relevant images linked to a reasoning system for a close comparison with the analyzed material, and a reference system with several modules: correlated image and text databases; descriptions of particular diagnoses; differential diagnosis facility for comparing two different diagnostic hypotheses; references to relevant literature on particular diagnoses. This article presents our approach to designing a consulting system and adapting it to a tutoring environment. We concentrate on the knowledge acquisition stage and identification of different classes of users likely to use a system with analysis of their requirements and the types of knowledge they bring to bear on a problem. We explain the background and approach that we have taken to design the inferencing mechanism operating like an automated theorem prover....