This paper extends the currently accepted model of inductive bias by identifying six categories of bias and separates inductive bias from the policy for its selection (the inductive policy). We analyze existing "blas selection " systems, examining the similarities and differences in their inductive policies, and idemify three techniques useful for building inductive policies. We then present a framework for representing and automaticaIly selecting a wide variety of biases and describe experiments with an instantiation of the framework addressing various pragmatic tradeoffs of time, space, accuracy, and the cost oferrors. The experiments show that a common framework can be used to implement policies for a variety of different types of blas selection, such as parameter selection, term selection, and example selection, using similar techniques. The experiments also show that different tradeoffs can be made by the implementation of different policies; for example, from the same data different rule sets can be learned based on different tradeoffs of accuracy versus the cost of erroneous predictions.

. Interest in Declarative bias in Machine Learning is growing with the expressivity of the concept description language of ML systems. Inductive Logic Programming more than any other ML field is thus concerned with explicitely biasing learning. The main issues already identified in declarative bias [RG90] have been studied within the ILP project, i.e. the restriction of the size of the search space for the target concept and representation of the bias. As a first step, an extensive study of existing ILP systems and the elicitation of the role of hidden bias has led to define typologies of bias in relation with their effects on the learning process as well as alternative representation for bias. Declarative representations of bias have been defined through different types of languages so that bias can be easily set and shifted. In parallel with the definition, the representation and the experimentation of various biases, the interactions between different types of bias have been analyze...

Because the knowledge discovery process is ill-defined, iterative, and requires intense interaction, algorithm flexibility is crucial. In this paper, we present a straighforward, heuristic generate-and-test search algorithm for knowledge discovery. An analysis of the literature shows that this basic algorithm underlies many of the systems that have had practical success in data mining and knowledge discovery over the past twenty years. We argue that this search algorithm has persevered because it is flexible and well behaved as background knowledge is introduced in various forms - exactly what is needed to support the ill-defined knowledge discovery process.

We describe an incremental learning algorithm, called theory-driven learning, that creates rules to predict the effect of actions. Theory-driven learning exploits knowledge of regularities among rules to constrain learning. We demonstrate that this knowledge enables the learning system to rapidly converge on accurate predictive rules and to tolerate more complex training data. An algorithm for incrementally learning these regularities is described and we provide evidence that the resulting regularities are sufficiently general to facilitate learning in new domains. The results demonstrate transfer from one domain to another can be achieved by deliberately overgeneralizing rules in one domain and biasing the learning algorithm to create new rules that specialize these overgeneralizations in other domains.

. Machine Learning (ML) methods are very powerful tools to automate the knowledge acquisition (KA) task. Particularly, in illstructured domains where there is no clear idea about which concepts exist, inductive unsupervised learning systems appear to be a promising approach to help experts in the early stages of the acquisition process. In this paper we examine the concept of inductive bias, which have received great attention from the ML community, and discuss the importance of bias shift when using ML algorithms to help experts in constructing a knowledge base (KB) A simple framework for the interaction of the expert with the inductive system exploiting bias shift is shown. Also, it is suggested that under some assumptions, bias selection in unsupervised learning may be performed via parameter setting, thus allowing the user to shift the system bias externally. 1 Introduction One of the original goals in developing Machine Learning (ML) programs was to overcome the well-known probl...

We suggest why, and show how, to represent defeasible reasoning about prioritization-type precedence. We define Defeasible Axiomatized Policy (DAP) circumscription: it is the first formalism to express defeasible prioritization. DAP circumscription can represent one or more (generally, a finite reflective tower) of meta-levels of such reasoning, without resorting to a more powerful logical language. We argue for the usefulness, and analyze the expressive significance, of this representational generalization. We show that it can often be achieved with only a modest increase in the mathematical complexity of inference: DAP circumscription often reduces to a series of prioritized predicate circumscriptions, for which inference procedures are currently available. DAP circumscription also offers an improved approach to pointwise prioritization and circumscription, even in the basic, monotonic case of reasoning about prioritization. We observe that unsatisfiability and representational awkw...