## Simultaneous Variable and Transformation Selection in Linear Regression (1995)

Venue: | JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS |

Citations: | 7 - 5 self |

### BibTeX

@TECHREPORT{Hoeting95simultaneousvariable,

author = {Jennifer A. Hoeting and Adrian E. Raftery and David Madigan},

title = {Simultaneous Variable and Transformation Selection in Linear Regression},

institution = {JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS},

year = {1995}

}

### Years of Citing Articles

### OpenURL

### Abstract

We suggest a method for simultaneous variable and transformation selection based on posterior probabilities. A simultaneous approach avoids the problem that the order in which they are done might influence the choice of variables and transformations. The simultaneous approach also allows for consideration of all possible models. We use a change-point model, or "change-point transformation", which often yields more interpretable models and transformations than the standard Box-Tidwell approach. We also address the problem of model uncertainty in the selection of models. By averaging over models, we account for the uncertainty inherent in inference based on a single model chosen from the set of all possible models. We use a Markov chain Monte Carlo model composition (MC³) method which allows us to average over linear regression models when the space of all possible models is very large. This considers the selection of variables and transformations at the same time. In an example, we ...

### Citations

984 | Bayes factors - Kass, Raftery - 1995 |

332 | Variable selection via gibbs sampling - George, McCulloch - 1993 |

265 | Models selection and accounting for model uncertainty in graphical models using occam’s window
- Madigan, Raftery
- 1994
(Show Context)
Citation Context ...e selection, outlier identification, and transformation selection. As an alternative to MC 3 , a procedure called Occam's Window can be used to select models to include in the Bayesian model average (=-=Madigan and Raftery 1994-=-; Raftery et al. 17 1994). By reducing the model space over which the model average is computed, computation of the required posterior model probabilities becomes computationally feasible. We do not a... |

236 |
Estimating optimal transformations for multiple regression and correlation
- Breiman, Friedman
- 1985
(Show Context)
Citation Context ...ange point transformations". We use a two-step process to identify a change point transformation for a predictor. First we use the output from the Alternating Conditional Expectation (ACE) algori=-=thm (Breiman and Friedman 1985-=-) to suggest the form of the transformation. In the second, confirmatory stage, we use Bayes factors to choose the precise transformation to be considered. The ACE algorithm provides nonlinear transfo... |

226 | Bayesian graphical models for discrete data - Madigan, York - 1995 |

186 | Bayesian model averaging for linear regression models - Raftery, Madigan, et al. - 1997 |

183 |
Specification Searches: Ad Hoc Inference with Nonexperimental Data
- Leamer
- 1978
(Show Context)
Citation Context ...g to a single "best" model and to then make inferences as if the selected model were the true model. However, this ignores a major component of uncertainty, namely uncertainty about the mode=-=l itself (Leamer 1978-=-, Raftery 1993, Draper 1995). As a consequence, uncertainty about quantities of interest can be underestimated. For striking examples of this see Regal and Hook (1991), Raftery (1993) and Kass and Raf... |

155 | Rational decisions - Good - 1952 |

111 |
Assessment and propagation of model uncertainty (with discussion
- Draper
- 1995
(Show Context)
Citation Context ...and to then make inferences as if the selected model were the true model. However, this ignores a major component of uncertainty, namely uncertainty about the model itself (Leamer 1978, Raftery 1993, =-=Draper 1995-=-). As a consequence, uncertainty about quantities of interest can be underestimated. For striking examples of this see Regal and Hook (1991), Raftery (1993) and Kass and Raftery (1995). The standard B... |

98 | Approximate Bayes Factors and Accounting for Model Uncertainty in Generalized Linear Regression Models. Biometrika
- Raftery
- 1996
(Show Context)
Citation Context ... "best" model and to then make inferences as if the selected model were the true model. However, this ignores a major component of uncertainty, namely uncertainty about the model itself (Lea=-=mer 1978, Raftery 1993-=-, Draper 1995). As a consequence, uncertainty about quantities of interest can be underestimated. For striking examples of this see Regal and Hook (1991), Raftery (1993) and Kass and Raftery (1995). T... |

82 | An analysis of transformations (with discussion - Box, Cox - 1964 |

72 |
Predictive Inference: An Introduction
- Geisser
- 1993
(Show Context)
Citation Context ...e of statistical analysis is to make forecasts for the future, and several authors have advocated making predictive performance the main criterion for evaluating statistical methods (e.g. Dawid 1984; =-=Geisser 1993-=-; Hahn and Meeker 1993). Here we compare the quality of the predictions from model averaging with that of the predictions from any single model that an analyst might reasonably have selected. To measu... |

50 | Interactive elicitation of opinion for a normal linear model - Kadane, Dickey, et al. - 1980 |

40 | Theory of Probability (3rd - Jeffreys - 1961 |

17 | Elicitation of Prior Distributions for Variable-Selection Problems in Regression Annals of Statistics - Garthwaite, Dickey - 1992 |

15 | Selection of subsets of regression variables (with Discussion - Miller - 1984 |

11 | Model Selection for Generalized linear model via GLIB: Application to nutrition and breast cancer - AE, Richardson - 1996 |

11 | The effects of model selection on confidence intervals for the size of a closed population - Regal, Hook - 1991 |

10 | Demand or ideation? Evidence from the Iranian marital fertility decline,” Demography 32(2 - Raftery, Lewis, et al. - 1995 |

7 |
Accounting for model uncertainty in linear regression
- Hoeting
- 1994
(Show Context)
Citation Context ...d, computation of the required posterior model probabilities becomes computationally feasible. We do not apply Occam's Window in this paper, but we have successfully applied it in the context of SVT (=-=Hoeting, 1994-=-). Draper (1995) has also addressed the problem of assessing model uncertainty. Draper's approach is based on the idea of model expansion, i.e., starting with a single reasonable model chosen by a dat... |

6 |
Statistical theory{the prequential approach
- Dawid
- 1984
(Show Context)
Citation Context ...imary purpose of statistical analysis is to make forecasts for the future, and several authors have advocated making predictive performance the main criterion for evaluating statistical methods (e.g. =-=Dawid 1984-=-; Geisser 1993; Hahn and Meeker 1993). Here we compare the quality of the predictions from model averaging with that of the predictions from any single model that an analyst might reasonably have sele... |

4 | Comments on "Sampling and Bayes' inference in scientific modeling and robustness"JRSS-A - Geisser - 1980 |

4 | Event history modeling of World Fertility Survey data." Working Paper No - Raftery, Lewis, et al. - 1993 |

1 | Prediction via Orthogonalized Model Mixing. Unpublished manuscript - Clyde, DeSimone, et al. - 1994 |

1 |
Change Point and Change Curve Modeling
- Raftery
- 1994
(Show Context)
Citation Context ... outlier identification, and transformation selection. As an alternative to MC 3 , a procedure called Occam's Window can be used to select models to include in the Bayesian model average (Madigan and =-=Raftery 1994-=-; Raftery et al. 17 1994). By reducing the model space over which the model average is computed, computation of the required posterior model probabilities becomes computationally feasible. We do not a... |