Abstract

It is argued that P-values and the tests based upon them give unsatisfactory results, especially in large samples. It is shown that, in regression, when there are many candidate independent variables, standard variable selection procedures can give very misleading results. Also, by selecting a single model, they ignore model uncertainty and so underestimate the uncertainty about quantities of interest. The Bayesian approach to hypothesis testing, model selection and accounting for model uncertainty is presented. Implementing this is straightforward using the simple and accurate BIC approximation, and can be done using the output from standard software. Specific results are presented for most of the types of model commonly used in sociology. It is shown that this approach overcomes the difficulties with P values and standard model selection procedures based on them. It also allows easy comparison of non-nested models, and permits the quantification of the evidence for a null hypothesis...

Citations

717	Bayes factors - Kass, Raftery - 1995
215	Model selection and accounting for model uncertainty in graphical models using Occam's window - Madigan, Raftery - 1994
113	Rational Decisions - Good - 1952
86	Bayesianly justifiable and relevant frequency calculations for the applied statistician - Rubin - 1984
80	Participation in illegitimate activities: a theoretical and empirical investigation”, The - Ehrlich - 1973
79	Approximate Bayes Factors and Accounting for Model Uncertainty in Generalised Linear Models - Raftery - 1996
70	Sellke T: Testing a point null hypothesis: The irreconcilability of P-values and evidence - Berger - 1987
37	Accounting for model uncertainty in survival analysis improves predictive performance (with discussion - Raftery, Madigan, et al. - 1996
22	The Comparative Method - Ragin - 1987
16	Testing precise hypotheses (with discussion - DELAMPADY - 1987
15	Bowhead whale, Balaena mysticetus, population size estimated from acoustic and visual census data collected near - Raftery, Zeh - 1994
14	Bayesian Decision Theory and the Simplification of Models," in Evaluation of Econometric Models - Kadane, Dickey - 1980
12	A reference Bayesian test for nested hypotheses with large samples - Kaass, Wasserman - 1995
10	What is a Good Job? A New Measure of Labor Market Success - Jencks, Perman, et al. - 1988
9	Comparitive social mobility revisited: Models of convergence and divergence in 16 countries - Grusky, Hauser - 1984
8	Predicting working memory failure: A subjective Bayesian approach to model selection - Carlin, Kass, et al. - 1992
7	Bayesian model checking using tail area probability", Statistica Sinica (with discussion - Gelman, Meng, et al. - 1995
5	Scaling the Intergenerational Continuity of Occupation: Is Occupational Inheritance Ascriptive After All - Rytina - 1992
4	A note on screening regression equations." The American Statistician 37 - Freedman - 1983
3	Counter examples to parsimony and BIC - Findley - 1991
1	Some general points in probability theory. Pp. 451--454 in Bayesian Analysis in Econometrics and Statistics, edited by Arnold Zellner - Jeffreys - 1980