The present paper deals with monotonic and dual monotonic language learning from positive as well as from positive and negative examples. The three notions of monotonicity reflect different formalizations of the requirement that the learner has to produce better and better generalizations when fed more and more data on the concept to be learned.

In designing learning algorithms it seems quite reasonable to construct them in a way such that all data the algorithm already has obtained are correctly and completely reflected in the hypothesis the algorithm outputs on these data. However, this approach may totally fail, i.e., it may lead to the unsolvability of the learning problem, or it may exclude any efficient solution of it. In particular, we present a natural learning problem and prove that it can be solved in polynomial time if and only if the algorithm is allowed to ignore data.

page 14 Declaration - page 15 Notes of copyright and the ownership of intellectual property rights - page 15 The Author - page 16 Acknowledgements - page 16 1 - Introduction - page 17 1.1 - Background - page 17 1.2 - The Style of Approach - page 18 1.3 - Motivation - page 19 1.4 - Style of Presentation - page 20 1.5 - Outline of the Thesis - page 21 2 - Models and Modelling - page 23 2.1 - Some Types of Models - page 25 2.2 - Combinations of Models - page 28 2.3 - Parts of the Modelling Apparatus - page 33 2.4 - Models in Machine Learning - page 38 2.5 - The Philosophical Background to the Rest of this Thesis - page 41 Syntactic Measures of Complexity - page 3 - 3 - Problems and Properties - page 44 3.1 - Examples of Common Usage - page 44 3.1.1 - A case of nails - page 44 3.1.2 - Writing a thesis - page 44 3.1.3 - Mathematics - page 44 3.1.4 - A gas - page 44 3.1.5 - An ant hill - page 45 3.1.6 - A car engine - page 45 3.1.7 - A cell as part of an organism -...

Kleene's Second Recursion Theorem provides a means for transforming any program p into a program e(p) which first creates a quiescent self copy and then runs p on that self copy together with any externally given input. e(p), in effect, has complete (low level) self knowledge, and p represents how e(p) uses its self knowledge (and its knowledge of the external world). Infinite regress is not required since e(p) creates its self copy outside of itself. One mechanism to achieve this creation is a self replication trick isomorphic to that employed by single-celled organisms. Another is for e(p) to look in a mirror to see which program it is. In 1974 the author published an infinitary generalization of Kleene's theorem which he called the Operator Recursion Theorem. It provides a means for obtaining an (algorithmically) growing collection of programs which, in effect, share a common (also growing) mirror from which they can obtain complete low level models of themselves and the other prog...

this paper we assume, without loss of generality, that for all oe ` ø , [M(oe) 6=?] ) [M(ø) 6=?].

Two learning situations are considered: machine identification of programs from graphs of recursive functions (modeling inductive hypothesis formation) and machine identification of grammars from texts of recursively enumerable languages (modeling first language acquisition). Both these learning models are extended to account for situations in which a learning machine is provided additional information in the form of knowledge about an upper-bound on the minimal size program (grammar) for the function (language) being identified. For a number of such extensions, it is shown that larger classes of functions (languages) can be algorithmically identified in the presence of upper-bound information. Numerous interesting relationships are shown between different models of learning, number of anomalies allowed in the inferred program (grammar), and number of anomalies allowed in the upper-bound information. 1

We study what kind of data may ease the computational complexity of learning of Horn clause theories (in Gold's paradigm) and Boolean functions (in PAC-learning paradigm). We give several definitions of good data (basic and generative representative sets), and develop data-driven algorithms that learn faster from good examples, and degenerate to learn in the limit from the "worst" possible examples. We show that Horn clause theories, kterm DNF and general DNF Boolean functions are polynomially learnable from generative representative presentations. 1 Introduction In any inductive learning model, how data of the target theory are supplied to the learning programs is a crucial assumption. Identification in the limit [ Gold, 1967 ] assumes that the series of examples is an admissible enumeration of all (positive and/or negative) examples of the target concept, and requires the learning algorithm to produce a correct hypothesis in some finite time. However, the computational time and th...

A team of learning machines is essentially a multiset of learning machines.

The present paper deals with the learnability of indexed families of uniformly recursive languages from positive data under various postulates of naturalness. In particular, we consider set-driven and rearrangement-independent learners, i.e., learning devices whose output exclusively depends on the range and on the range and length of their input, respectively. The impact of set-drivenness and rearrangement-independence on the behavior of learners to their learning power is studied in dependence on the hypothesis space the learners may use. Furthermore, we consider the influence of set-drivenness and rearrangementindependence for learning devices that realize the subset principle to different extents. Thereby we distinguish between strong-monotonic, monotonic and weak-monotonic or conservative learning. The results obtained are twofold. First, rearrangement-independent learning does not constitute a restriction except the case of monotonic learning. Second, we prove that for all but on...

We extend Angluin's (1980) theorem to characterize identifiability of indexed families of r.e. languages, as opposed to indexed families of recursive languages. We also prove some variants characterizing conservativity and two other similar restrictions, paralleling Zeugmann, Lange, and Kapur's (1992, 1995) results for indexed families of recursive languages. 1 Introduction A significant portion of the work of recent years in the field of inductive inference of formal languages, as initiated by Gold 1967, stems from Angluin's (1980b) theorem, which characterizes when an indexed family of recursive languages is identifiable in the limit from positive data in the sense of Gold. Up until around 1980, a prevalent view had been that inductive inference from positive data is too weak to be of much theoretical interest. This misconception was due to the negative result in Gold's original paper, which says that any class of languages that contains every finite language and at least one infini...