Abstract

. A Machine can only learn if it is biased in some way. Typically the bias is supplied by hand, for example through the choice of an appropriate set of features. However, if the learning machine is embedded within an environment of related tasks, then it can learn its own bias by learning sufficiently many tasks from the environment [4, 6]. In this paper two models of bias learning (or equivalently, learning to learn) are introduced and the main theoretical results presented. The first model is a PAC-type model based on empirical process theory, while the second is a hierarchical Bayes model. Keywords: Learning to Learn, Bias Learning, Empirical Processes, Hierarchical Bayes 1. Introduction Hume's analysis [10] shows that there is no a priori basis for induction. In a machine learning context, this means that a learner must be biased in some way for it to generalise well [11]. Typically such bias is introduced by hand through the skill and insights of experts, but despite many notable...

Citations

687	Estimation of dependences based on empirical data - Vapnik - 1982
266	A Treatise of Human Nature - Hume - 1962
95	Computational learning theory - Anthony, Biggs - 1992
92	Information–theoretic asymptotics of Bayes methods - Clarke, Barron - 1990
81	Learning internal representations - Baxter - 1995
55	A bayesian/information theoretic model of learning to learn via multiple task sampling - Baxter - 1997
23	Some History of the Hierarchical Bayesian Methodology," in Good Thinking: The Foundations of Probability and its Applications - GOOD - 1983
10	The need for biases in learning generalisations - Mitchell - 1980
6	Jeffreys' Prior is Asymptotically Least Favourable under Entropy Risk - Clarke, Barron - 1994
4	A model for bias learning - Baxter - 1997
2	Multivariate Estimation - Berger - 1986
1	Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities - N - 1971