A new view of learning is presented. The basis of this view is a natural notion of reduction. We prove completeness and relative difficulty results. An infinite hierarchy of intrinsically more and more difficult to learn concepts is presented. Our results indicate that the complexity notion captured by our new notion of reduction differs dramatically from the traditional studies of the complexity of the algorithms performing learning tasks. 2 1 Introduction Traditional studies of inductive inference have focused on illuminating various strata of learnability based on varying the definition of learnability. The research following the Valiant's PAC model [Val84] and Angluin's teacher/learner model [Ang88] paid very careful attention to calculating the complexity of the learning algorithm. We present a new view of learning, based on the notion of reduction, that captures a different perspective on learning complexity than all prior studies. Based on our prelimanary reports, Jain...

this paper contributes toward the goal of understanding how a computer can be programmed to learn by isolating features of incremental learning algorithms that theoretically enhance their learning potential. In particular, we examine the effects of imposing a limit on the amount of information that learning algorithm can hold in its memory as it attempts to This work was facilitated by an international agreement under NSF Grant 9119540.

After the initial relevance of deduction in AI, nowadays, inductive learning, and all its varieties (abductive reasoning, reasoning by analogy, connectionism, ILP, grammatical inference, HMM, EBR, etc.), are beginning to play a more central and still agglutinative role in the proper subset of AI devoted to make intelligent machines. The issue has been much clearer in discovery science, where induction has been the prominent inference process. In this trend, new unified frameworks for understanding reasoning have been appearing, with the aim of integrating all the different inference mechanims [25]. In particular, a radical approach has been undertaken by Wolff, with the claim that “all kinds of computing and formal reasoning may usefully be understood as information compression by pattern matching, unification and search ” [46]. In this paper, we will discuss critically a very influential and now classical issue in this line, which is based on the view of unsupervised learning as compression [38], its famous operative Minimum Description Length (MDL)