Results 1  10
of
12
Graphical Model Structure Learning with `1Regularization
, 2010
"... This work looks at fitting probabilistic graphical models to data when the structure is not known. The main tool to do this is `1regularization and the more general group `1regularization. We describe limitedmemory quasiNewton methods to solve optimization problems with these types of regularize ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
This work looks at fitting probabilistic graphical models to data when the structure is not known. The main tool to do this is `1regularization and the more general group `1regularization. We describe limitedmemory quasiNewton methods to solve optimization problems with these types of regularizers, and we examine learning directed acyclic graphical models with `1regularization, learning undirected graphical models with group `1regularization, and learning hierarchical loglinear models with overlapping group `1regularization. ii
1 Group Variable Selection Methods and Their Applications in Analysis of Genomic Data
"... Regression is a simple but the most useful statistical method in data analysis. The goal of regression analysis is to discover the relationship between a response y and a set of predictors x1, x2,..., xp. When fitting a regression model, besides prediction accuracy, parsimony is another important cr ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Regression is a simple but the most useful statistical method in data analysis. The goal of regression analysis is to discover the relationship between a response y and a set of predictors x1, x2,..., xp. When fitting a regression model, besides prediction accuracy, parsimony is another important criterion
Greedy and Relaxed Approximations to Model Selection: A simulation study
, 2008
"... The Minimum Description Length (MDL) principle is an important tool for retrieving knowledge from data as it embodies the scientific strife for simplicity in describing the relationship among variables. As MDL and other model selection criteria penalize models on their dimensionality, the estimation ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
The Minimum Description Length (MDL) principle is an important tool for retrieving knowledge from data as it embodies the scientific strife for simplicity in describing the relationship among variables. As MDL and other model selection criteria penalize models on their dimensionality, the estimation problem involves a combinatorial search over subsets of predictors and quickly becomes computationally cumbersome. Two approximation frameworks are: convex relaxation and greedy algorithms. In this article, we perform extensive simulations comparing two algorithms for generating candidate models that mimic the best subsets of predictors for given sizes (Forward Stepwise and the Least Absolute Shrinkage and Selection Operator LASSO). From the list of models determined by each method, we consider estimates chosen by two different model selection criteria (AICc and the generalized MDL criterion gMDL). The comparisons are made in terms of their selection and prediction performances. In terms of variable selection, we consider two different metrics. For the number of selection errors, our results suggest that the combination Forward Stepwise+gMDL has a better performance over different sample sizes and sparsity regimes. For the second metric of rate of true positives among the selected variables, LASSO+gMDL seems more appropriate for very small sample sizes, while Forward Stepwise+gMDL has a better performance for sample sizes at least as large as the number of factors being screened. Moreover, we found that, asymptotically, Zhao and Yu’s ((1)) irrepresentibility condition (index) has a larger impact on the selection performance of Lasso than on Forward Stepwise. In what refers to prediction performance, LASSO+AICc results in good predictive models over a wide range of sample sizes and sparsity regimes. Last but not least, these simulation results reveal that one method often can not serve for both selection and prediction purposes. 1
ADVANCES IN VARIABLE SELECTION AND VISUALIZATION METHODS FOR ANALYSIS OF MULTIVARIATE DATA
, 2007
"... ..."
(Show Context)
Regression by enhanced leapsandbounds via additional optimality tests (LBOT).” Submitted, 2005. Available from the authors
"... In exhaustive subset selection in regressions, the leapsandbounds algorithm by Furnival and Wilson (1974) is the current stateoftheart. It utilizes a branch and bound strategy. We improve it by introducing newly designed optimality tests, retaining the original general framework. Being compared ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
In exhaustive subset selection in regressions, the leapsandbounds algorithm by Furnival and Wilson (1974) is the current stateoftheart. It utilizes a branch and bound strategy. We improve it by introducing newly designed optimality tests, retaining the original general framework. Being compared with the original leapsandbounds algorithm, the proposed method further reduces the number of subsets that are needed to be considered in the exhaustive subset search. Simulations demonstrate the improvements in numerical performance. Our new description of the leapsandbounds algorithm, which is based on our newly designed pair tree, is independent of programming languages, and therefore is more accessible. AMS 2000 Subject Classification. Primary 62J05, secondary 62F07.
Bayesian
"... feature selection for highdimensional linear regression via the Ising approximation with applications to genomics ..."
Abstract
 Add to MetaCart
(Show Context)
feature selection for highdimensional linear regression via the Ising approximation with applications to genomics
and
"... Abstract: We show that with a class of penalty functions, numerical problems associated with the implementation of the penalized least square estimators are equivalent to the exact cover by 3sets problem, which belongs to a class of NPhard problems. We then extend this NPhardness result to the ca ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract: We show that with a class of penalty functions, numerical problems associated with the implementation of the penalized least square estimators are equivalent to the exact cover by 3sets problem, which belongs to a class of NPhard problems. We then extend this NPhardness result to the cases of penalized least absolute deviation regression and penalized support vector machines. We discuss the practical implication of our results. In particular, we emphasize that the oracle property of a penalized likelihood estimator requires a local extremum, instead of a global extremum. Hence the penalized likelihood estimators are still favorable; however one should not attempt to find its global extremum(a)!