Results 1  10
of
27
Consistent Model Selection Criteria on High Dimensions
"... Asymptotic properties of model selection criteria for highdimensional regression models are studied where the dimension of covariates is much larger than the sample size. Several sufficient conditions for model selection consistency are provided. NonGaussian error distributions are considered and ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Asymptotic properties of model selection criteria for highdimensional regression models are studied where the dimension of covariates is much larger than the sample size. Several sufficient conditions for model selection consistency are provided. NonGaussian error distributions are considered and it is shown that the maximal number of covariates for model selection consistency depends on the tail behavior of the error distribution. Also, sufficient conditions for model selection consistency are given when the variance of the noise is neither known nor estimated consistently. Results of simulation studies as well as real data analysis are given to illustrate that finite sample performances of consistent model selection criteria can be quite different.
Shrinkage Tuning Parameter Selection in Precision Matrices Estimation
, 909
"... Recent literature provides many computational and modeling approaches for covariance matrices estimation in a penalized Gaussian graphical models but relatively little study has been carried out on the choice of the tuning parameter. This paper tries to fill this gap by focusing on the problem of sh ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Recent literature provides many computational and modeling approaches for covariance matrices estimation in a penalized Gaussian graphical models but relatively little study has been carried out on the choice of the tuning parameter. This paper tries to fill this gap by focusing on the problem of shrinkage parameter selection when estimating sparse precision matrices using the penalized likelihood approach. Previous approaches typically used Kfold crossvalidation in this regard. In this paper, we first derived the generalized approximate crossvalidation for tuning parameter selection which is not only a more computationally efficient alternative, but also achieves smaller error 1 rate for model fitting compared to leaveoneout crossvalidation. For consistency in the selection of nonzero entries in the precision matrix, we employ a Bayesian information criterion which provably can identify the nonzero conditional correlations in the Gaussian model. Our simulations demonstrate the general superiority of the two proposed selectors in comparison with leaveoneout crossvalidation, tenfold crossvalidation and Akaike information criterion.
A Near Minimax Risk Bound: Adaptive Lasso with Heteroskedastic Data In Instrumental Variable Selection
, 2011
"... In this paper we use adaptive lasso estimator select between relevant and irrelevant instruments in heteroskedastic and non Gaussian data. To do so limit theory of Zou (2006) is extended from univariate iid case. Next, it is shown that adaptive lasso estimator can achieve near minimax risk bound eve ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
In this paper we use adaptive lasso estimator select between relevant and irrelevant instruments in heteroskedastic and non Gaussian data. To do so limit theory of Zou (2006) is extended from univariate iid case. Next, it is shown that adaptive lasso estimator can achieve near minimax risk bound even in the case of heteroskedastic data. To achieve that a new proof is used that benefits from Stein’s Lemma. This is a new result and extends the iid Gaussian case. It is also shown in the paper that Lasso estimators are not model selection consistent whereas adaptive lasso can select the correct model in fixed number of instruments case. The case of weak versus strong instruments are also handled by adaptive lasso. Simulations show that compared to alternatives in econometrics it does well in terms of bias.
An Alternative to Unit Root Tests: Bridge Estimators Differentiate between Nonstationary versus Stationary Models and Select Optimal Lag Mehmet Caner
, 2012
"... This paper introduces a novel way of differentiating a unit root from stationary alternatives using socalled “Bridge ” estimators; this estimation procedure can potentially generate exact zero estimates of parameters. We exploit this property and treat this as a model selection problem. We show tha ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
This paper introduces a novel way of differentiating a unit root from stationary alternatives using socalled “Bridge ” estimators; this estimation procedure can potentially generate exact zero estimates of parameters. We exploit this property and treat this as a model selection problem. We show that Bridge estimators can select the correct model with probability tending to 1. They estimate ”zero ” parameter on the lagged dependent variable as zero (nonstationarity), if this is nonzero (stationary), estimate the coefficient with standard normal limit. In this sense, we extend the statistics literature as well, since that literature only deals with model selection among only stationary variables. The reason that our methodology can outperform the existing unit root tests with lag selection methods stems from the twostep nature of existing unit root tests. In our method, we select the optimal lag length and unit root simultaneously. We show that in simulations, this makes a substantial difference in terms of size and power.
Penalized highdimensional empirical likelihood
 Biometrika
, 2005
"... We propose the penalized empirical likelihood (PEL) method for parameter estimation and variable selection for problems with diverging numbers of parameters. Our results are demonstrated for estimating the mean vector in multivariate analysis and regression coefficients in linear models. By using an ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
We propose the penalized empirical likelihood (PEL) method for parameter estimation and variable selection for problems with diverging numbers of parameters. Our results are demonstrated for estimating the mean vector in multivariate analysis and regression coefficients in linear models. By using an appropriate penalty function, we show that PEL has the oracle property. That is, with probability tending to one, PEL identifies the true model and estimates the nonzero coefficients as efficiently as if the sparsity of the true model were known in advance. The advantage of PEL as a nonparametric likelihood approach is illustrated in testing hypothesis and constructing confidence sets. Numerical simulations confirm our theoretical findings.
Adaptive Elastic Net for Generalized Methods of Moments Mehmet Caner North Carolina State University ∗
, 2013
"... Model selection and estimation are crucial parts of econometrics. This paper introduces a new technique that can simultaneously estimate and select the model in generalized method of moments (GMM) context. The GMM is particularly powerful for analyzing complex data sets such as longitudinal and pane ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Model selection and estimation are crucial parts of econometrics. This paper introduces a new technique that can simultaneously estimate and select the model in generalized method of moments (GMM) context. The GMM is particularly powerful for analyzing complex data sets such as longitudinal and panel data, and it has wide applications in econometrics. This paper extends the least squares based adaptive elastic net estimator of Zou and Zhang (2009) to nonlinear equation systems with endogenous variables. The extension is not trivial and involves a new proof technique due to estimators lack of closed form solutions. Compared to BridgeGMM of Caner (2009), we allow for the number of parameters to diverge to infinity as well as collinearity among a large number of variables, also the redundant parameters set to zero via a data dependent technique. This method has the oracle property, meaning that we can estimate nonzero parameters with their standard limit and the redundant parameters are dropped from the equations simultaneously. Numerical examples are used to illustrate the performance of the new method.
Adaptive Elastic Net GMM Estimation with Many Invalid Moment Conditions: Simultaneous Model and Moment Selection
, 2015
"... This paper develops the adaptive elastic net GMM estimator in large dimensional models with potentially (locally) invalid moment conditions, where both the number of structural parameters and the number of moment conditions may increase with the sample size. The basic idea is to conduct the standard ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
This paper develops the adaptive elastic net GMM estimator in large dimensional models with potentially (locally) invalid moment conditions, where both the number of structural parameters and the number of moment conditions may increase with the sample size. The basic idea is to conduct the standard GMM estimation combined with two penalty terms: the adaptively weighted lasso shrinkage and the quadratic regularization. It is a onestep procedure of valid moment condition selection, nonzero structural parameter selection (i.e., model selection), and consistent estimation of the nonzero parameters. The procedure achieves the standard GMM efficiency bound as if we know the valid moment conditions ex ante, for which the quadratic regularization is important. We also study the tuning parameter choice, with which we show that selection consistency still holds without assuming Gaussianity. We apply the new estimation procedure to dynamic panel data models, where both the time and cross section dimensions are large. The new estimator is robust to possible serial correlations in the regression error terms.
A LASSOpenalized BIC for mixture model selection
 Advances in Data Analysis and Classification
, 2014
"... The efficacy of familybased approaches to mixture modelbased clustering and classification depends on the selection of parsimonious models. Current wisdom suggests the Bayesian information criterion (BIC) for mixture model selection. However, the BIC has wellknown limitations, including a tendenc ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
The efficacy of familybased approaches to mixture modelbased clustering and classification depends on the selection of parsimonious models. Current wisdom suggests the Bayesian information criterion (BIC) for mixture model selection. However, the BIC has wellknown limitations, including a tendency to overestimate the number of components as well as a proclivity for, often drastically, underestimating the number of components in higher dimensions. While the former problem might be soluble through merging components, the latter is impossible to mitigate in clustering and classification applications. In this paper, a LASSOpenalized BIC (LPBIC) is introduced to overcome this problem. This approach is illustrated based on applications of extensions of mixtures of factor analyzers, where the LPBIC is used to select both the number of components and the number of latent factors. The LPBIC is shown to match or outperform the BIC in several situations. 1
Flexible Shrinkage Estimation in HighDimensional Varying Coefficient Models.” Working Paper
"... ar ..."
(Show Context)
Consistent Selection of Tuning Parameters via Variable Selection Stability
"... Penalized regression models are popularly used in highdimensional data analysis to conduct variable selection and model fitting simultaneously. Whereas success has been widely reported in literature, their performances largely depend on the tuning parameters that balance the tradeoff between mod ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Penalized regression models are popularly used in highdimensional data analysis to conduct variable selection and model fitting simultaneously. Whereas success has been widely reported in literature, their performances largely depend on the tuning parameters that balance the tradeoff between model fitting and model sparsity. Existing tuning criteria mainly follow the route of minimizing the estimated prediction error or maximizing the posterior model probability, such as cross validation, AIC and BIC. This article introduces a general tuning parameter selection criterion based on variable selection stability. The key idea is to select the tuning parameters so that the resultant penalized regression model is stable in variable selection. The asymptotic selection consistency is established for both fixed and diverging dimensions. Its effectiveness is also demonstrated in a variety of simulated examples as well as an application to the prostate cancer data.