Abstract

In this paper we introduce a new sparseness inducing prior which does not involve any (hyper) parameters that need to be adjusted or estimated. Although other applications are possible, we focus here on supervised learning problems: regression and classification. Experiments with several publicly available benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms support vector machines and performs competitively with the best alternative techniques, both in terms of error rates and sparseness, although it involves no tuning or adjusting of sparsenesscontrolling hyper-parameters.

Citations

6696	The Nature of Statistical Learning Theory - Vapnik - 1995
3909	Neural Networks for Pattern Recognition - Bishop - 1995
1089	Atomic decomposition by basis pursuit - Chen, Donoho, et al. - 1998
1057	An Introduction to Support Vector Machines and Other Kernel-based Learning Methods - Cristianini, Shawe-Taylor - 2000
998	Regression shrinkage and selection via the lasso - Tibshirani - 1996
944	J: General Linear Models - McCullagh, Nelder - 1989
917	Statistical decision theory and Bayesian analysis - Berger - 1985
896	BD: Pattern Recognition and Neural Networks - Ripley - 1996
570	Networks for approximation and learning - Poggio, Girosi - 1990
513	Bayesian Learning for Neural Networks - Neal - 1996
327	Ridge regression: Biased estimation for nonorthogonal problems - Hoerl, Kennard - 2000
291	Bayesian analysis of binary and polychotomous response data - Albert, Chib - 1993
207	Using the nyström method to speed up kernel machines - Williams, Seeger
193	Learning from Data: Concepts, Theory, and Methods - Cherkassky, Mulier - 1998
169	The Relevance Vector Machine - Tipping - 2000
167	An equivalence between sparse approximation and support vector machines - Girosi - 1998
160	Prediction with Gaussian processes: From linear regression to linear prediction and beyond - Williams - 1998
121	A new approach to variable selection in least squares problems - Osborne, Presnell, et al. - 2000
110	A correspondence between Bayesian estimation on stochastic processes and smoothing by splines - Kimeldorf, Wahba - 1970
58	Bayesian regularization and pruning using a Laplace prior - Williams - 1995
39	Wavelet-Based image estimation: An empirical Bayes approach using Jeffreys' y noninformative prior - Figueiredo, Nowak - 2001
38	Bayesian model selection for support vector machines, Gaussian processes and other kernel classifiers - Seeger - 2000
16	Bayesian non-linear modelling for the 1993 energy prediction competition - MacKay - 1996
15	Ideal adaptation via wavelet shrinkage - Donoho, Johnstone - 1994
13	Normal/independent distributions and their applications in robust regression - Lange, Sinsheimer - 1993
6	Bayesian classification with Gaussian priors - Williams, Barber - 1998