DMCA
Sparse Bayesian Learning and the Relevance Vector Machine (2001)
| Citations: | 963 - 5 self |
BibTeX
@MISC{Tipping01sparsebayesian,
author = {Michael E. Tipping},
title = {Sparse Bayesian Learning and the Relevance Vector Machine},
year = {2001}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper introduces a general Bayesian framework for obtaining sparse solutions to regression and classification tasks utilising models linear in the parameters. Although this framework is fully general, we illustrate our approach with a particular specialisation that we denote the `relevance vector machine’ (RVM), a model of identical functional form to the popular and state-of-the-art `support vector machine ’ (SVM). We demonstrate that by exploiting a probabilistic Bayesian learning framework, we can derive accurate prediction models which typically utilise dramatically fewer basis functions than a comparable SVM while offering a number of additional advantages. These include the benefits of probabilistic predictions, automatic estimation of `nuisance’ parameters, and the facility to utilise arbitrary basis functions (e.g. non-`Mercer’ kernels). We detail the Bayesian framework and associated learning algorithm for the RVM, and give some illustrative examples of its application along with some comparative benchmarks. We offer some explanation for the exceptional degree of sparsity obtained, and discuss and demonstrate some of the advantageous features, and potential extensions, of Bayesian relevance learning.
Keyphrases
relevance vector machine sparse bayesian learning nuisance parameter comparable svm probabilistic prediction general bayesian framework comparative benchmark particular specialisation arbitrary basis function advantageous feature sparse solution exceptional degree accurate prediction model bene t automatic estimation mercer kernel illustrative example bayesian relevance learning potential extension support vector machine probabilistic bayesian learning framework additional advantage learning algorithm bayesian framework identical functional form basis function classi cation task utilising model
