Abstract

Three Bayesian ideas are presented for supervised adaptive classifiers. First, it is argued that the output of a classifier should be obtained by marginalising over the posterior distribution of the parameters; a simple approximation to this integral is proposed and demonstrated. This involves a `moderation' of the most probable classifier 's outputs, and yields improved performance. Second, it is demonstrated that the Bayesian framework for model comparison described for regression models in (MacKay, 1992a, 1992b) can also be applied to classification problems. This framework successfully chooses the magnitude of weight decay terms, and ranks solutions found using different numbers of hidden units. Third, an information--based data selection criterion is derived and demonstrated within this framework. 1 Introduction A quantitative Bayesian framework has been described for learning of mappings in feedforward networks (MacKay, 1992a, 1992b). It was demonstrated that this `evidence' fram...

Citations

417	Bayesian interpolation - MacKay - 1992
346	A practical Bayesian framework for back-propagation networks - MacKay - 1992
245	Learning and relearning in Boltzmann machines - Hinton, Sejnowski - 1986
236	Information-Based Objective Functions for Active Data - MacKay - 1992
194	Probalistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition - Bridle - 1990
133	Bayesian Methods for Adaptive Models - MacKay - 1992
59	Neural Net Algorithms that Learn in Polynomial Time from Examples and Queries - Baum - 1991
49	Accelerated Learning in Layered Neural Networks - Solla, Levin, et al. - 1988
40	Exact calculation of the hessian matrix for the multilayer perceptron - Bishop - 1992
35	Query-based learning applied to partially trained multilayer perceptrons - Hwang, Choi, et al. - 1991
29	Learning algorithms and probability distributions in feed-forward networks - Hopfield
10	Active Selection of Training Examples for Network Learning in Noiseless Environments - Plutowski, White - 1991
1	Statistical mechanics of learning from examples', preprint - Seung, Sompolinsky, et al. - 1991