Abstract

The support vector machine (SVM) is a state-of-the-art technique for regression and classification, combining excellent generalisation properties with a sparse kernel representation. However, it does suffer from a number of disadvantages, notably the absence of probabilistic outputs, the requirement to estimate a trade-off parameter and the need to utilise `Mercer' kernel functions. In this paper we introduce the Relevance Vector Machine (RVM), a Bayesian treatment of a generalised linear model of identical functional form to the SVM. The RVM suffers from none of the above disadvantages, and examples demonstrate that for comparable generalisation performance, the RVM requires dramatically fewer kernel functions.

Citations

6694	Statistical Learning Theory - Vapnik - 1998
917	Statistical decision theory and Bayesian analysis - Berger - 1985
513	Bayesian Learning for Neural Networks - Neal - 1996
417	Bayesian interpolation - MacKay - 1992
60	Input space versus feature space in kernelbased methods - Scholkopf, Mika, et al. - 1999
38	DJC: Bayesian non-linear modelling for the prediction competition ASHRAE Transactions - MacKay - 1994
16	Bayesian classi with Gaussian processes - Williams, Barber - 1998
7	The evidence framework applied to classi networks - MacKay - 1992
5	Ecient training of RBF networks for classi - Nabney - 1999