Results 1  10
of
151
Regularization and variable selection via the Elastic Net
 Journal of the Royal Statistical Society, Series B
, 2005
"... Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where ..."
Abstract

Cited by 360 (8 self)
 Add to MetaCart
Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the p n case. An algorithm called LARSEN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lasso.
Consensus clustering  A resamplingbased method for class discovery and visualization of gene expression microarray data
 MACHINE LEARNING, FUNCTIONAL GENOMICS SPECIAL ISSUE
, 2003
"... ..."
Sparse graphical models for exploring gene expression data
 Journal of Multivariate Analysis
, 2004
"... DMS0112069. Any opinions, findings, and conclusions or recommendations expressed in this material are ..."
Abstract

Cited by 132 (22 self)
 Add to MetaCart
DMS0112069. Any opinions, findings, and conclusions or recommendations expressed in this material are
Correcting sample selection bias by unlabeled data
"... We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appropriate corrections based on the distribution estimate. We prese ..."
Abstract

Cited by 130 (9 self)
 Add to MetaCart
We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appropriate corrections based on the distribution estimate. We present a nonparametric method which directly produces resampling weights without distribution estimation. Our method works by matching distributions between training and testing sets in feature space. Experimental results demonstrate that our method works well in practice.
BagBoosting for tumor classification with gene expression data
 Bioinformatics
, 2004
"... Motivation: Microarray experiments are expected to contribute significantly to the progress in cancer treatment by enabling a precise and early diagnosis. They create a need for class prediction tools, which can deal with a large number of highly correlated input variables, perform feature selection ..."
Abstract

Cited by 126 (2 self)
 Add to MetaCart
Motivation: Microarray experiments are expected to contribute significantly to the progress in cancer treatment by enabling a precise and early diagnosis. They create a need for class prediction tools, which can deal with a large number of highly correlated input variables, perform feature selection and provide class probability estimates that serve as a quantification of the predictive uncertainty. A very promising solution is to combine the two ensemble schemes bagging and boosting to a novel algorithm called BagBoosting.
Results: When bagging is used as a module in boosting, the resulting classifier consistently improves the predictive performance and the probability estimates of both bagging and boosting on real and simulated gene expression data. This quasiguaranteed improvement can be obtained by simply making a bigger computing effort. The advantageous predictive potential is also confirmed by comparing BagBoosting to several established class prediction tools for microarray data.
Bayesian Factor Regression Models in the "Large p, Small n" Paradigm
 Bayesian Statistics
, 2003
"... TOR REGRESSION MODELS 1.1 SVD Regression Begin with the linear model y = X# + # where y is the nvector of responses, X is the n p matrix of predictors, # is the pvector regression parameter, and # , # I) is the nvector error term. Of key interest are cases when p >> n, when X is "long a ..."
Abstract

Cited by 108 (13 self)
 Add to MetaCart
TOR REGRESSION MODELS 1.1 SVD Regression Begin with the linear model y = X# + # where y is the nvector of responses, X is the n p matrix of predictors, # is the pvector regression parameter, and # , # I) is the nvector error term. Of key interest are cases when p >> n, when X is "long and skinny." The standard empirical factor (principal component) regression is best represented using the reduced singularvalue decomposition (SVD) of X, namely X = FA where F is the nk factor matrix (columns are factors, rows are samples) and A is the k p SVD "loadings" matrix, subject to AA # = I and F # F = D where D is the diagonal matrix of k positive singular values, arranged in decreasing order. This reduced form assumes factors with zero singular values have been ignored without loss; k with equality only if all singular values are positive. Now the regression transforms via X# = F# where # = A# is the kvector of regression parameters for the factor variables, representing
Boosting for highdimensional linear models
 THE ANNALS OF STATISTICS
, 2006
"... We prove that boosting with the squared error loss, L2Boosting, is consistent for very highdimensional linear models, where the number of predictor variables is allowed to grow essentially as fast as O(exp(sample size)), assuming that the true underlying regression function is sparse in terms of th ..."
Abstract

Cited by 39 (5 self)
 Add to MetaCart
We prove that boosting with the squared error loss, L2Boosting, is consistent for very highdimensional linear models, where the number of predictor variables is allowed to grow essentially as fast as O(exp(sample size)), assuming that the true underlying regression function is sparse in terms of the ℓ1norm of the regression coefficients. In the language of signal processing, this means consistency for denoising using a strongly overcomplete dictionary if the underlying signal is sparse in terms of the ℓ1norm. We also propose here an AICbased method for tuning, namely for choosing the number of boosting iterations. This makes L2Boosting computationally attractive since it is not required to run the algorithm multiple times for crossvalidation as commonly used so far. We demonstrate L2Boosting for simulated data, in particular where the predictor dimension is large in comparison to sample size, and for a difficult tumorclassification problem with gene expression microarray data.
Boosting algorithms: Regularization, prediction and model fitting
 Statistical Science
, 2007
"... Abstract. We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and correspo ..."
Abstract

Cited by 38 (5 self)
 Add to MetaCart
Abstract. We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in highdimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated opensource software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing userspecified loss functions. Key words and phrases: Generalized linear models, generalized additive models, gradient boosting, survival analysis, variable selection, software. 1.
Effective dimension reduction methods for tumor classification using gene expression data
 Bioinformatics
, 2003
"... Motivation: One particular application of microarray data, is to uncover the molecular variation among cancers. One feature of microarray studies is the fact that the number n of samples collected is relatively small compared to the number p of genes per sample which are usually in the thousands. In ..."
Abstract

Cited by 35 (2 self)
 Add to MetaCart
Motivation: One particular application of microarray data, is to uncover the molecular variation among cancers. One feature of microarray studies is the fact that the number n of samples collected is relatively small compared to the number p of genes per sample which are usually in the thousands. In statistical terms this very large number of predictors compared to a small number of samples or observations makes the classification problem difficult. An efficient way to solve this problem is by using dimension reduction statistical techniques in conjunction with nonparametric discriminant procedures. Results: We view the classification problem as a regression problem with few observations and many predictor variables. We use an adaptive dimension reduction method for generalized semiparametric regression models that allows us to solve the ‘curse of dimensionality problem ’ arising in the context of expression data. The predictive performance of the resulting classification rule is illustrated on two well know data sets in the microarray literature: the leukemia data that is known to contain classes that are easy ‘separable ’ and the colon data set. Availability: Software that implements the procedures on which this paper focus are freely available at