Abstract

Probably the most important problem in machine learning is the preliminary biasing of a learner's hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt. In this paper a mechanism for automatically learning or biasing the learner's hypothesis space is introduced. It works by first learning an appropriate internal representation for a learning environment and then using that representation to bias the learner's hypothesis space for the learning of future tasks drawn from the same environment. An internal representation must be learnt by sampling from many similar tasks, not just a single task as occurs in ordinary machine learning. It is proved that the number of examples m per task required to ensure good generalisation from a representation learner obeys m = O(a+b=n) where n is the number of tasks being learnt and a and b are constants. If the tasks are l...

Citations

583	Learning representations by back-propagating errors - Rumelhart, Hinton, et al. - 1986
537	Neural networks and the bias/variance dilemma - Geman, Bienenstock, et al. - 1992
457	Convergence of Stochastic Processes - Pollard - 1984
328	Decision theoretic generalization of the PAC model for neural net and other learning applications - Haussler - 1992