Abstract

Model averaging provides an alternative to model selection. An algorithm ARM rooted in information theory is proposed to combine different regression models/methods. A simulation is conducted in the context of linear regression to compare its performance with familiar model selection criteria AIC and BIC, and also with some Bayesian model averaging (BMA) methods. The simulation suggests

Citations

1998	Bagging predictors - Breiman - 1996
840	Information theory and an extension of the maximum likelihood principle - Akaike - 1973
717	Bayes factors - Kass, Raftery - 1995
556	The weighted majority algorithm - Littlestone, Warmuth - 1994
264	Classification and Regression - Breiman, Friedman, et al. - 1984
235	Estimating the dimension of a model - Schwartz - 1978
213	Model selection and accounting for model uncertainty in graphical models using Occam’s window - Madigan, Raftery - 1994
211	Aggregating strategies - Vovk - 1990
177	AE: Bayesian Model Selection in Social Research - Raftery - 1996
176	Minimum complexity density estimation - Barron, Cover - 1991
132	Bayesian model averaging for linear regression models - Raftery, Madigan, et al. - 1997
99	The Intrinsic Bayes Factor for Model Selection and Prediction - Berger, Perrichi - 1996
80	Participation in illegitimate activities: a theoretical and empirical investigation”, The - Ehrlich - 1973
79	Assessment and propagation of model uncertainty - Draper - 1995
75	Approaches for Bayesian variable selection - George, McCulloch - 1997
67	Information-theoretic determination of minimax rates of convergence - Yang, Barron - 1999
61	Benchmark priors for Bayesian model averaging - Fernández, Ley, et al. - 2001
28	Mixing strategies for density estimation - Yang
27	Are bayes rules consistent in information - Barron - 1987
26	Model selection: an integral part of inference - Buckland, Burnhamn, et al. - 1997
22	Functional aggregation for nonparametric estimation - Juditsky, Nemirovski - 2000
21	A mixture approach to universal model selection - Catoni - 1997
19	Adaptive regression by mixing - Yang - 2001
7	Bridging AIC and BIC: An MDL model selection criterion - Hansen, Yu - 1999
6	Combing estimates in regression and classi cation - LeBlanc, Tibshirani - 1996
5	Asymptotically optimal functional estimation by minimum complexity criteria - Barron - 1994
5	How to use expert advise - Cesa-Bianchi, Freund, et al. - 1997
1	A.E.andVolinsky, C.T.(1999)Bayesian model averaging: A tutotial - Hoeting, Madigan, et al.
1	Bayesian graphical models for discreate data - Madigan, York - 1995
1	2000b) Combining di erent regression procedures for adaptive regression - Yang