Results

**1 - 6**of**6**### ADVANCES IN VARIABLE SELECTION AND VISUALIZATION METHODS FOR ANALYSIS OF MULTIVARIATE DATA

, 2007

"... ..."

### and

"... Abstract: We show that with a class of penalty functions, numerical problems associated with the implementation of the penalized least square estimators are equivalent to the exact cover by 3-sets problem, which belongs to a class of NP-hard problems. We then extend this NP-hardness result to the ca ..."

Abstract
- Add to MetaCart

Abstract: We show that with a class of penalty functions, numerical problems associated with the implementation of the penalized least square estimators are equivalent to the exact cover by 3-sets problem, which belongs to a class of NP-hard problems. We then extend this NP-hardness result to the cases of penalized least absolute deviation regression and penalized support vector machines. We discuss the practical implication of our results. In particular, we emphasize that the oracle property of a penalized likelihood estimator requires a local extremum, instead of a global extremum. Hence the penalized likelihood estimators are still favorable; however one should not attempt to find its global extremum(a)!

### Chinese Academy of Sciences, Beijing 100190

"... Since the penalized likelihood function of the smoothly clipped absolute deviation (SCAD) penalty is highly non-linear and has many local optima, finding a local solution to achieve the so-called oracle property is an open problem. We propose an iterative algorithm, called the OEM algorithm, to fill ..."

Abstract
- Add to MetaCart

Since the penalized likelihood function of the smoothly clipped absolute deviation (SCAD) penalty is highly non-linear and has many local optima, finding a local solution to achieve the so-called oracle property is an open problem. We propose an iterative algorithm, called the OEM algorithm, to fill this gap. The development of the algorithm draws direct impetus from a missing-data problem arising in design of experiments with an orthogonal complete matrix. In each iteration, the algorithm imputes the missing data based on the current estimates of the parameters and updates a closed-form solution associated with the complete data. By introducing a procedure called active orthogonization, we make the algorithm broadly applicable to problems with arbitrary regression matrices. In addition to the SCAD penalty, the proposed algorithm works for other penalties like the MCP, lasso and nonnegative garrote. Convergence and convergence rate of the algorithm are examined. The algorithm has several unique theoretical properties. For the SCAD and MCP penalties, an OEM sequence can achieve the oracle property after sufficient iterations. For various penalties, an OEM sequence converges to a point having grouping coherence for fully aliased regression matrices. For computing the ordinary least squares estimator with a singular regression matrix, an OEM sequence converges to the Moore-Penrose generalized inverse-based least squares estimator.

### Greedy and Relaxed Approximations to Model Selection: A simulation study

, 2008

"... The Minimum Description Length (MDL) principle is an important tool for retrieving knowledge from data as it embodies the scientific strife for simplicity in describing the relationship among variables. As MDL and other model selection criteria penalize models on their dimensionality, the estimation ..."

Abstract
- Add to MetaCart

The Minimum Description Length (MDL) principle is an important tool for retrieving knowledge from data as it embodies the scientific strife for simplicity in describing the relationship among variables. As MDL and other model selection criteria penalize models on their dimensionality, the estimation problem involves a combinatorial search over subsets of predictors and quickly becomes computationally cumbersome. Two approximation frameworks are: convex relaxation and greedy algorithms. In this article, we perform extensive simulations comparing two algorithms for generating candidate models that mimic the best subsets of predictors for given sizes (Forward Stepwise and the Least Absolute Shrinkage and Selection Operator- LASSO). From the list of models determined by each method, we consider estimates chosen by two different model selection criteria (AICc and the generalized MDL criterion- gMDL). The comparisons are made in terms of their selection and prediction performances. In terms of variable selection, we consider two different metrics. For the number of selection errors, our results suggest that the combination Forward Stepwise+gMDL has a better performance over different sample sizes and sparsity regimes. For the second metric of rate of true positives among the selected variables, LASSO+gMDL seems more appropriate for very small sample sizes, while Forward Stepwise+gMDL has a better performance for sample sizes at least as large as the number of factors being screened. Moreover, we found that, asymptotically, Zhao and Yu’s ((1)) irrepresentibility condition (index) has a larger impact on the selection performance of Lasso than on Forward Stepwise. In what refers to prediction performance, LASSO+AICc results in good predictive models over a wide range of sample sizes and sparsity regimes. Last but not least, these simulation results reveal that one method often can not serve for both selection and prediction purposes. 1

### 1 Group Variable Selection Methods and Their Applications in Analysis of Genomic Data

"... Regression is a simple but the most useful statistical method in data analysis. The goal of regression analysis is to discover the relationship between a response y and a set of predictors x1, x2,..., xp. When fitting a regression model, besides prediction accuracy, parsimony is another important cr ..."

Abstract
- Add to MetaCart

Regression is a simple but the most useful statistical method in data analysis. The goal of regression analysis is to discover the relationship between a response y and a set of predictors x1, x2,..., xp. When fitting a regression model, besides prediction accuracy, parsimony is another important criterion