Results 1  10
of
21
Onestep sparse estimates in nonconcave penalized likelihood models. Ann. Statist., to appear. 36 Proof of Theorems 2(ii) and 4 Proof of Theorem 2(ii). To prove asymptotic normality for ˆφ n1, note that by (A.23), for αn with ‖αn‖ = 1 and νn = αnHnαn, n 1
 n1) = I1 + I2 + I3, (S.1) where I2 = λn(nνn) −1/2 α T n G−1 11 Wns/2 , I3
, 2008
"... Fan and Li propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective funct ..."
Abstract

Cited by 58 (0 self)
 Add to MetaCart
Fan and Li propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective function is nondifferentiable and nonconcave. In this article, we propose a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized likelihood for a broad class of concave penalty functions. Convergence and other theoretical properties of the LLA algorithm are established. A distinguished feature of the LLA algorithm is that at each LLA step, the LLA estimator can naturally adopt a sparse representation. Thus, we suggest using the onestep LLA estimator from the LLA algorithm as the final estimates. Statistically, we show that if the regularization parameter is appropriately chosen, the onestep LLA estimates enjoy the oracle properties with good initial estimators. Computationally, the onestep LLA estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood. We conduct some Monte Carlo simulation to assess the finite sample performance of the onestep sparse estimation methods. The results are very encouraging. 1. Introduction. Variable
Sparse image reconstruction for molecular imaging
 IEEE Trans. Image Process
, 2009
"... Abstract—The application that motivates this paper is molecular imaging at the atomic level. When discretized at subatomic distances, the volume is inherently sparse. Noiseless measurements from an imaging technology can be modeled by convolution of the image with the system point spread function (p ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
Abstract—The application that motivates this paper is molecular imaging at the atomic level. When discretized at subatomic distances, the volume is inherently sparse. Noiseless measurements from an imaging technology can be modeled by convolution of the image with the system point spread function (psf). Such is the case with magnetic resonance force microscopy (MRFM), an emerging technology where imaging of an individual tobacco mosaic virus was recently demonstrated with nanometer resolution. We also consider additive white Gaussian noise (AWGN) in the measurements. Many prior works of sparse estimators have focused on the case when H has low coherence; however, the system matrix H in our application is the convolution matrix for the system psf. A typical convolution matrix has high coherence. This paper, therefore, does not assume a low coherence H. A discretecontinuous form of the Laplacian and atom at zero (LAZE) p.d.f. used by Johnstone and Silverman is formulated, and two sparse estimators derived by maximizing the joint p.d.f. of the observation and image conditioned on the hyperparameters. A thresholding rule that generalizes the hard and soft thresholding rule appears in the course of the derivation. This socalled hybrid thresholding rule, when used in the iterative thresholding framework, gives rise to the hybrid estimator, a generalization of the lasso. Estimates of the hyperparameters for the lasso and hybrid estimator are obtained via Stein’s unbiased risk estimate (SURE). A numerical study with a Gaussian psf and two sparse images shows that the hybrid estimator outperforms the lasso. Index Terms—Biomedical image processing, image restoration, magnetic force microscopy, sparse image prior, Stein’s unbiased risk estimate.
Nonnegative Garrote Component Selection in Functional ANOVA Models
, 2007
"... We consider the problem of component selection in a functional ANOVA model. A nonparametric extension of the nonnegative garrote (Breiman, 1996) is proposed. We show that the whole solution path of the proposed method can be efficiently computed, which, in turn, facilitates the selection of the tuni ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
We consider the problem of component selection in a functional ANOVA model. A nonparametric extension of the nonnegative garrote (Breiman, 1996) is proposed. We show that the whole solution path of the proposed method can be efficiently computed, which, in turn, facilitates the selection of the tuning parameter. We also show that the final estimate enjoys nice theoretical properties given that the tuning parameter is appropriately chosen. Simulation and a real data example demonstrate promising performance of the new approach.
Bayesian generalized double Pareto shrinkage
, 2010
"... We propose a generalized double Pareto prior for shrinkage estimation in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, while forming a bridge between the Laplace and NormalJeffreys ’ priors. While it has a spike at zero like the Laplace density, it ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
We propose a generalized double Pareto prior for shrinkage estimation in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, while forming a bridge between the Laplace and NormalJeffreys ’ priors. While it has a spike at zero like the Laplace density, it also has a Studenttlike tail behavior. We show strong consistency of the posterior in regression models with a diverging number of parameters, providing a template to be used for other priors in similar settings. Bayesian computation is straightforward via a simple Gibbs sampling algorithm. We also investigate the properties of the maximum a posteriori estimator and reveal connections with some wellestablished regularization procedures. The performance of the new prior is tested through simulations.
Feature Selection via BlockRegularized Regression
"... Identifying covarying causal elements in very high dimensional feature space with internal structures, e.g., a space with as many as millions of linearly ordered features, as one typically encounters in problems such as whole genome association (WGA) mapping, remains an open problem in statistical ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Identifying covarying causal elements in very high dimensional feature space with internal structures, e.g., a space with as many as millions of linearly ordered features, as one typically encounters in problems such as whole genome association (WGA) mapping, remains an open problem in statistical learning. We propose a blockregularized regression model for sparse variable selection in a highdimensional space where the covariates are linearly ordered, and are possibly subject to local statistical linkages (e.g., block structures) due to spacial or temporal proximity of the features. Our goal is to identify a small subset of relevant covariates that are not merely from random positions in the ordering, but grouped as contiguous blocks from large number of ordered covariates. Following a typical linear regression framework between the features and the response, our proposed model employs a sparsityenforcing Laplacian prior for the regression coefficients, augmented by a 1storder Markovian process along the feature sequence that “activates” the regression coefficients in a coupled fashion. We describe a samplingbased learning algorithm and demonstrate the performance of our method on simulated and biological data for marker identification under WGA. 1
Penalized quadratic inference functions for variable selection in longitudinal research
, 2006
"... For decades, much research has been devoted to developing and comparing variable selection methods, but primarily for the classical case of independent observations. Existing variableselection methods can be adapted to clustercorrelated observations, but some adaptation is required. For example, ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
For decades, much research has been devoted to developing and comparing variable selection methods, but primarily for the classical case of independent observations. Existing variableselection methods can be adapted to clustercorrelated observations, but some adaptation is required. For example, classical model fit statistics such as AIC and BIC are undefined if the likelihood function is unknown (Pan, 2001). Little research has been done on variable selection for generalized estimating equations (GEE, Liang and Zeger, 1986) and similar correlated data approaches. This thesis will review existing work on model selection for GEE and propose new model selection options for GEE, as well as for a more sophisticated marginal modeling approach based on quadratic inference functions (QIF, Qu, Lindsay, and Li, 2000), which has better asymptotic properties than classic GEE. The focus is on selection using continuous penalties such as LASSO (Tibshirani, 1996) or SCAD (Fan and Li, 2001) rather than the older discrete penalties such as AIC and BIC. The
Fast Bayesian Model Assessment for Nonparametric Additive Regression ✩
"... Variable selection techniques for the classical linear regression model have been widely investigated. Variable selection in fully nonparametric and additive regression models have been studied more recently. A Bayesian approach for nonparametric additive regression models is considered, where the f ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Variable selection techniques for the classical linear regression model have been widely investigated. Variable selection in fully nonparametric and additive regression models have been studied more recently. A Bayesian approach for nonparametric additive regression models is considered, where the functions in the additive model are expanded in a Bspline basis and a multivariate Laplace prior is put on the coefficients. Posterior probabilities of models defined by selection of predictors in the working model are computed, using a Laplace approximation method. The prior times the likelihood is expanded around the posterior mode, which can be identified with the group LASSO, for which a fast computing algorithm exists. Thus Markov chain MonteCarlo or any other time consuming sampling based methods are completely avoided, leading to quick assessment of various posterior model probabilities. This technique is applied to the highdimensional situation where the number of parameters exceeds the number of observations. Keywords:
On the Nonnegative Garrote Estimator 1
, 2005
"... We study the nonnegative garrote estimator from three different aspects: computation, consistency and flexibility. We show that the nonnegative garrote estimate has a piecewise linear solution path. Using this fact, we propose an efficient algorithm for computing the whole solution path for the nonn ..."
Abstract
 Add to MetaCart
We study the nonnegative garrote estimator from three different aspects: computation, consistency and flexibility. We show that the nonnegative garrote estimate has a piecewise linear solution path. Using this fact, we propose an efficient algorithm for computing the whole solution path for the nonnegative garrote estimate. We also show that the nonnegative garrote has the nice property that with probability tending to one, the solution path contains an estimate that correctly identifies the set of important variables and is consistent for the coefficients of the important variables. Such property is valid for another popular variable selection method, LASSO, only under restrictive conditions. We propose a slight modification that retains the attractive properties of the original nonnegative garrote, but is more widely applicable. To demonstrate the flexibility of the proposed estimator, we consider an extension to the nonparametric regression setup. Simulations and a real example show that the proposed method is very competitive in terms of variable selection and estimation accuracy when compared with other variable selection and estimation methods.
An Efficient Variable Selection Approach for Analyzing Designed Experiments
"... The analysis of experiments where a large number of potential variables are examined is driven by the principles of effect sparsity, effect hierarchy, and effect heredity. We propose an efficient variable selection strategy to specifically address the unique challenges faced by such analysis. The pr ..."
Abstract
 Add to MetaCart
The analysis of experiments where a large number of potential variables are examined is driven by the principles of effect sparsity, effect hierarchy, and effect heredity. We propose an efficient variable selection strategy to specifically address the unique challenges faced by such analysis. The proposed methods are natural extensions of a generalpurpose variable selection algorithm, LARS (Efron et al., 2004). They are very fast to compute and can find sparse models that better satisfy the goals of experiments. Simulations and real examples are used to illustrate the wide applicability of the proposed methods. 1