Results 1  10
of
30
Mixtures of gpriors for Bayesian variable selection
 Journal of the American Statistical Association
, 2008
"... Zellner’s gprior remains a popular conventional prior for use in Bayesian variable selection, despite several undesirable consistency issues. In this paper, we study mixtures of gpriors as an alternative to default gpriors that resolve many of the problems with the original formulation, while mai ..."
Abstract

Cited by 36 (4 self)
 Add to MetaCart
Zellner’s gprior remains a popular conventional prior for use in Bayesian variable selection, despite several undesirable consistency issues. In this paper, we study mixtures of gpriors as an alternative to default gpriors that resolve many of the problems with the original formulation, while maintaining the computational tractability that has made the gprior so popular. We present theoretical properties of the mixture gpriors and provide real and simulated examples to compare the mixture formulation with fixed gpriors, Empirical Bayes approaches and other default procedures.
The Horseshoe Estimator for Sparse Signals
, 2008
"... This paper proposes a new approach to sparsity called the horseshoe estimator. The horseshoe is a close cousin of other widely used Bayes rules arising from, for example, doubleexponential and Cauchy priors, in that it is a member of the same family of multivariate scale mixtures of normals. But th ..."
Abstract

Cited by 21 (6 self)
 Add to MetaCart
This paper proposes a new approach to sparsity called the horseshoe estimator. The horseshoe is a close cousin of other widely used Bayes rules arising from, for example, doubleexponential and Cauchy priors, in that it is a member of the same family of multivariate scale mixtures of normals. But the horseshoe enjoys a number of advantages over existing approaches, including its robustness, its adaptivity to different sparsity patterns, and its analytical tractability. We prove two theorems that formally characterize both the horseshoe’s adeptness at large outlying signals, and its superefficient rate of convergence to the correct estimate of the sampling density in sparse situations. Finally, using a combination of real and simulated data, we show that the horseshoe estimator corresponds quite closely to the answers one would get by pursuing a full Bayesian modelaveraging approach using a discrete mixture prior to model signals and noise.
Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction
, 2010
"... We use Lévy processes to generate joint prior distributions for a location parameter β = (β1,..., βp) as p grows large. This approach, which generalizes normal scalemixture priors to an infinitedimensional setting, has a number of connections with mathematical finance and Bayesian nonparametrics. ..."
Abstract

Cited by 16 (5 self)
 Add to MetaCart
We use Lévy processes to generate joint prior distributions for a location parameter β = (β1,..., βp) as p grows large. This approach, which generalizes normal scalemixture priors to an infinitedimensional setting, has a number of connections with mathematical finance and Bayesian nonparametrics. We argue that it provides an intuitive framework for generating new regularization penalties and shrinkage rules; for performing asymptotic analysis on existing models; and for simplifying proofs of some classic results on normal scale mixtures.
Generalized SURE for exponential families: Applications to regularization
 IEEE Trans. on Signal Processing
, 2009
"... ..."
Comparing between estimation approaches: Admissible and dominating linear estimators
 August 2005, EE Dept., Technion–Israel Institute of Technology
"... We treat the problem of evaluating the performance of linear estimators for estimating a deterministic parameter vector x in a linear regression model, with the meansquared error (MSE) as the performance measure. Since the MSE depends on the unknown vector x, direct comparison between estimators is ..."
Abstract

Cited by 13 (11 self)
 Add to MetaCart
We treat the problem of evaluating the performance of linear estimators for estimating a deterministic parameter vector x in a linear regression model, with the meansquared error (MSE) as the performance measure. Since the MSE depends on the unknown vector x, direct comparison between estimators is a difficult problem. Here we consider a framework for examining the MSE of different linear estimation approaches based on the concepts of admissible and dominating estimators. We develop a general procedure for determining whether or not a linear estimator is MSE admissible, and for constructing an estimator strictly dominating a given inadmissible method, so that its MSE is smaller for all x. In particular we show that both problems can be addressed in a unified manner for arbitrary constraint sets on x by considering a certain convex optimization problem. We then demonstrate the details of our method for the case in which x is constrained to an ellipsoidal set, and for unrestricted choices of x. As a by product of our results, we derive a closed form solution for the minimax MSE estimator on an ellipsoid, which is valid for arbitrary model parameters, as long as the signaltonoiseratio exceeds a certain threshold. Key Words—Linear estimation, regression, admissible estimators, dominating estimators, meansquared error (MSE) estimation, minimax MSE estimation.
Empirical Bayes vs. fully Bayes variable selection
, 2008
"... For the problem of variable selection for the normal linear model, fixed penalty selection criteria such as AIC, Cp, BIC and RIC correspond to the posterior modes of a hierarchical Bayes model for various fixed hyperparameter settings. Adaptive selection criteria obtained by empirical Bayes estimati ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
For the problem of variable selection for the normal linear model, fixed penalty selection criteria such as AIC, Cp, BIC and RIC correspond to the posterior modes of a hierarchical Bayes model for various fixed hyperparameter settings. Adaptive selection criteria obtained by empirical Bayes estimation of the hyperparameters have been shown by George and Foster [2000. Calibration and Empirical Bayes variable selection. Biometrika 87(4), 731–747] to improve on these fixed selection criteria. In this paper, we study the potential of alternative fully Bayes methods, which instead margin out the hyperparameters with respect to prior distributions. Several structured prior formulations are considered for which fully Bayes selection and estimation methods are obtained. Analytical and simulation comparisons with empirical Bayes counterparts are studied.
A Prior for the Variance in Hierarchical Models
, 1998
"... The choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a 'noninformative' type prior is usually chosen. `Noninformative' priors have been discussed by ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
The choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a 'noninformative' type prior is usually chosen. `Noninformative' priors have been discussed by many authors and used in many contexts. However, care must be taken using these prior distributions as many are improper and thus, can lead to improper posterior distributions. Additionally, in small samples, these priors can be 'informative'. In this paper, we investigate a proper 'vague' prior, the uniform shrinkage prior (Strawderman, 1971; Christiansen and Morris, 1997). We discuss its properties and show how posterior distributions for common hierarchical models using this prior lead to proper posterior distributions. We also illustrate the attractive frequentist properties of this prior for a normal hierarchical model including testing and estimation. To conclude, we generalize this p...
Handling sparsity via the horseshoe
 Journal of Machine Learning Research, W&CP
"... This paper presents a general, fully Bayesian framework for sparse supervisedlearning problems based on the horseshoe prior. The horseshoe prior is a member of the family of multivariate scale mixtures of normals, and is therefore closely related to widely used approaches for sparse Bayesian learni ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
This paper presents a general, fully Bayesian framework for sparse supervisedlearning problems based on the horseshoe prior. The horseshoe prior is a member of the family of multivariate scale mixtures of normals, and is therefore closely related to widely used approaches for sparse Bayesian learning, including, among others, Laplacian priors (e.g. the LASSO) and Studentt priors (e.g. the relevance vector machine). The advantages of the horseshoe are its robustness at handling unknown sparsity and large outlying signals. These properties are justified theoretically via a representation theorem and accompanied by comprehensive empirical experiments that compare its performance to benchmark alternatives. 1