Abstract

The problem of variable selection is one of the most pervasive model selection problems in statistical applications. Often referred to as the problem of subset selection, it arises when one wants to model the relationship between a variable of interest and a subset of potential explanatory variables or predictors, but there is uncertainty about which subset to use. This vignette reviews some of the key developments which have led to the wide variety of approaches for this problem. 1

Citations

1583	Estimating the dimension of a model - Schwarz - 1978
1393	Controlling the false discovery rate: a practical and powerful approach to multiple testing - Benjamini, Hochberg - 1995
999	Regression shrinkage and selection via the lasso - Tibshirani - 2006
891	Modeling by the Shortest Data Description - Rissanen - 1978
843	Information theory and an extension of the maximum likelihood principle - Akaike - 1973
578	Ideal spatial adaptation by wavelet shrinkage - Donoho - 1994
577	Reversible Jump Markov Chains Monte Carlo computation and Bayesian model determination - Green - 1995
498	Adapting to unknown smoothness via wavelet shrinkage - Donoho - 1995
230	Variable selection via Gibbs sampling - George, McCulloch - 1993
183	Subset Selection in Regression - Miller - 1990
150	Estimating the error rate of a prediction rule: improvement on cross-validation - EFRON - 1983
130	Some comments on Cp - Mallows - 1973
116	Regression and time series model selection in small samples - Hurvich, Tsai - 1989
106	Some comments on C p - Mallows - 1973
99	The Intrinsic Bayes Factor for Model Selection and Prediction - Berger, Perrichi - 1996
99	Assessment and propagation of model uncertainty (with discussion - Draper - 1995
97	Linear-Model Selection by Cross-Validation - Shao - 1993
94	A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion - Kass, Wasserman - 1995
91	Multiple shrinkage and subset selection in wavelets. Biometrika 85 391–401. MR1649120 - Clyde, Parmigiani, et al. - 1998
86	The risk inflation criterion for multiple regression - Foster, George - 1994
83	Regression by leaps and bounds - Furnival, Wilson - 1974
79	Approximate Bayes Factors and Accounting for Model Uncertainty in Generalised Linear Models - Raftery - 1996
77	An asymptotic equivalence of choice of model by crossvalidation and Akaike’s criterion - Stone - 1977
76	Fractional Bayes Factors for Model Comparison (with discussion - O’Hagan - 1995
75	Approaches for Bayesian variable selection - George, McCulloch - 1997
74	Bayesian model averaging: a tutorial (with discussion). Statist. Sci. 14 382–401. Corrected version at http://www.stat.washington.edu/www/research/online/hoeting1999.pdf - Hoeting, Madigan, et al. - 1999
66	The analysis and selection of variables in linear regression . Biometrics - Hocking - 1976
57	A predictive approach to model selection - Geisser, Eddy - 1979
54	H: Feature Selection for Classification. Intelligent Data Analysis - Dash, Liu - 1997
52	An optimal selection of regression variables - Shibata - 1981
50	On the choice of a model to fit data from an exponential family - Haughton - 1988
49	Flexible empirical Bayes estimation for wavelets - Clyde, George - 2000
49	Predictive Model Selection - Laud, Ibrahim - 1995
35	On measuring and correcting the effects of data mining and model selection - Ye - 1998
32	An asymptotic theory for linear model selection - Shao - 1997
31	Bayesian model averaging and model search strategies (with discussion - Clyde - 1999
28	Empirical bayes estimation in wavelet nonparametric regression - Clyde, George - 1999
23	Model selection via multifold cross-validation - Zhang - 1993
16	The Schwarz Criterion and Related Methods for Normal Linear Models - Pauler - 1998
15	Comments on model selection criteria of Akaike and - Stone - 1979
14	The choice of variables in multiple regression (with discussion - Lindley - 1968
13	Empirical Bayes approaches to mixture problems and wavelet regression", Technical report. Department of Mathematics, University of Bristol. Empirical Bayes Estimation in Wavelet Nonparametric Regression 15 - Johnstone, Silverman - 1998
12	Cross-Validation, the Jackknife, and the Bootstrap: Excess error estimation in forward logistic regression - Gong - 1986
12	A strongly consistent procedure for model selection in a regression problem - Rao, Wu - 1989
11	Selection of variables in multiple regression: Part I. A review and evaluation - Thompson - 1976
9	Multiple regression analysis. Mathematical methods for digital computers - Efroymson - 1960
8	Quantifying and using expert opinion for variable-selection problems in regression - Garthwaite, Dickey - 1996
8	A predictive model selection criterion - Martini, A, et al. - 1984
7	A Predictive Model Selection Criterion - SanMartini, Spezzaferri - 1984
6	Bayesian model selection. Encyclopedia of Statistical Sciences, Update Volume 3 - George - 1999