## On the Intrinsic Complexity of Learning (1995)

Venue: | Information and Computation |

Citations: | 25 - 6 self |

### BibTeX

@ARTICLE{Freivalds95onthe,

author = {Rusins Freivalds and Efim Kinber and Carl H. Smith},

title = {On the Intrinsic Complexity of Learning},

journal = {Information and Computation},

year = {1995},

volume = {123},

pages = {64--71}

}

### Years of Citing Articles

### OpenURL

### Abstract

A new view of learning is presented. The basis of this view is a natural notion of reduction. We prove completeness and relative difficulty results. An infinite hierarchy of intrinsically more and more difficult to learn concepts is presented. Our results indicate that the complexity notion captured by our new notion of reduction differs dramatically from the traditional studies of the complexity of the algorithms performing learning tasks. 2 1 Introduction Traditional studies of inductive inference have focused on illuminating various strata of learnability based on varying the definition of learnability. The research following the Valiant's PAC model [Val84] and Angluin's teacher/learner model [Ang88] paid very careful attention to calculating the complexity of the learning algorithm. We present a new view of learning, based on the notion of reduction, that captures a different perspective on learning complexity than all prior studies. Based on our prelimanary reports, Jain...

### Citations

1695 | A Theory of the Learnable
- Valiant
- 1984
(Show Context)
Citation Context ... Traditional studies of inductive inference have focused on illuminating various strata of learnability based on varying the definition of learnability. The research following the Valiant's PAC model =-=[Val84]-=- and Angluin's teacher/learner model [Ang88] paid very careful attention to calculating the complexity of the learning algorithm. We present a new view of learning, based on the notion of reduction, t... |

890 |
Language identification in the limit
- Gold
- 1967
(Show Context)
Citation Context ... given input from some linear time computable function, this procedure will, in the limit, converge to a correct program. The technique used is the above procedure is called the enumeration technique =-=[Gol67]-=-. In contrast with the simple learning procedure described above, consider learning the exponential time computable functions. In fact, the same basic algorithm works. The only modification necessary ... |

647 |
Queries and concept learning
- Angluin
- 1988
(Show Context)
Citation Context ...have focused on illuminating various strata of learnability based on varying the definition of learnability. The research following the Valiant's PAC model [Val84] and Angluin's teacher/learner model =-=[Ang88]-=- paid very careful attention to calculating the complexity of the learning algorithm. We present a new view of learning, based on the notion of reduction, that captures a different perspective on lear... |

259 | Toward a mathematical theory of inductive inference - Blum, Blum - 1975 |

163 |
Comparison of identification criteria for machine inductive inference, Theoretical Computer Science
- Case, Smith
- 1983
(Show Context)
Citation Context ...sses (and their derivatives) are ubiquitous in the study of inductive inference. The functions of finite support are known to also be learnable under the more restrictive notion of Popperian learning =-=[CS83]-=-. In fact, the self describing functions can be used as an example of something that is not in the more restrictive class containing the functions of finite support. To learn the self describing funct... |

101 | An Introduction to the General Theory of Algorithms - Machtey, Young - 1978 |

34 | On the role of procrastination in machine learning - Freivalds, Smith - 1993 |

26 | Two theorems on the limiting synthesis of functions - Barzdins - 1974 |

22 |
Reductions among prediction problems: on the difficulty of predicting automata (extended abstract
- Pitt, Warmuth
- 1988
(Show Context)
Citation Context ..., transforms samples of one concept C into samples of some other concept C 0 . The concept instances are represented by functions and sample points from the graph of the function serve as samples. In =-=[PW88]-=- the second transformation maps representations of concept C to representations of concept C 0 . In our study, the second mapping takes conjectures from attempts to learn a concept in C 0 to conjectur... |

21 | A Recursive Introduction to the Theory of Computation - Smith - 1994 |

19 | Learning via queries to an oracle - Gasarch, Pleszkoch - 1989 |

18 | Minimal GĂ¶del numbers and their identification in the limit - Freivalds - 1975 |

17 | On the intrinsic complexity of language identification - Jain, Sharma - 1994 |

15 |
Inductive inference of recursive functions: Qualitative theory
- Freivalds
- 1991
(Show Context)
Citation Context ...re called identification types. Our work entails a fresh perspective on learning. Consequently, there are several definition that serve to make more rigorous the intuitive notions described above. In =-=[Fre91]-=- the notion of an identification type was rigorously defined. Definition: An identification type is a predicate over two arguments one a recursive function and the other a sequence of programs. The se... |

15 | The structure of intrinsic complexity of learning - Jain, Sharma - 1995 |

10 | On the impact of forgetting on learning machines
- Freivalds, Kinber, et al.
- 1993
(Show Context)
Citation Context ...of finite support appear easy to learn is that deciding membership in the collection of "key" data is trivial. Furthermore, little effort is required in describing the function given the key=-= data. In [FKS93]-=- a lower bound on the amount of memory need to learn the functions of finite support was derived. This lower bound matches precisely the upper bound on memory requirements for any learning task. From ... |

7 | Towards representation independence in PAC-learning - Warmuth - 1989 |

3 | Inductive inference of minimal size programs - Freivalds - 1990 |

3 | Godel numberings of partial recursive functions - Jr - 1958 |

1 | Identification in the limit of minimal godel numbers - Freivalds, Kinber - 1977 |