Results 1  10
of
25
Posterior Predictive Assessment of Model Fitness Via Realized Discrepancies
 Statistica Sinica
, 1996
"... Abstract: This paper considers Bayesian counterparts of the classical tests for goodness of fit and their use in judging the fit of a single Bayesian model to the observed data. We focus on posterior predictive assessment, in a framework that also includes conditioning on auxiliary statistics. The B ..."
Abstract

Cited by 166 (28 self)
 Add to MetaCart
Abstract: This paper considers Bayesian counterparts of the classical tests for goodness of fit and their use in judging the fit of a single Bayesian model to the observed data. We focus on posterior predictive assessment, in a framework that also includes conditioning on auxiliary statistics. The Bayesian formulation facilitates the construction and calculation of a meaningful reference distribution not only for any (classical) statistic, but also for any parameterdependent “statistic ” or discrepancy. The latter allows us to propose the realized discrepancy assessment of model fitness, which directly measures the true discrepancy between data and the posited model, for any aspect of the model which we want to explore. The computation required for the realized discrepancy assessment is a straightforward byproduct of the posterior simulation used for the original Bayesian analysis. We illustrate with three applied examples. The first example, which serves mainly to motivate the work, illustrates the difficulty of classical tests in assessing the fitness of a Poisson model to a positron emission tomography image that is constrained to be nonnegative. The second and third examples illustrate the details of the posterior predictive approach in two problems: estimation in a model with inequality constraints on the parameters, and estimation in a mixture model. In all three examples, standard test statistics (either a χ 2 or a likelihood ratio) are not pivotal: the difficulty is not just how to compute the reference distribution for the test, but that in the classical framework no such distribution exists, independent of the unknown model parameters. Key words and phrases: Bayesian pvalue, χ 2 test, discrepancy, graphical assessment, mixture model, model criticism, posterior predictive pvalue, prior predictive
The interplay of bayesian and frequentist analysis
 Statist. Sci
, 2004
"... Statistics has struggled for nearly a century over the issue of whether the Bayesian or frequentist paradigm is superior. This debate is far from over and, indeed, should continue, since there are fundamental philosophical and pedagogical issues at stake. At the methodological level, however, the fi ..."
Abstract

Cited by 27 (0 self)
 Add to MetaCart
Statistics has struggled for nearly a century over the issue of whether the Bayesian or frequentist paradigm is superior. This debate is far from over and, indeed, should continue, since there are fundamental philosophical and pedagogical issues at stake. At the methodological level, however, the fight has become considerably muted, with the recognition that each approach has a great deal to contribute to statistical practice and each is actually essential for full development of the other approach. In this article, we embark upon a rather idiosyncratic walk through some of these issues. Key words and phrases: Admissibility; Bayesian model checking; conditional frequentist; confidence intervals; consistency; coverage; design; hierarchical models; nonparametric
Checking for priordata conflict
 Bayesian Analysis
, 2006
"... Abstract. Inference proceeds from ingredients chosen by the analyst and data. To validate any inferences drawn it is essential that the inputs chosen be deemed appropriate for the data. In the Bayesian context these inputs consist of both the sampling model and the prior. There are thus two possibil ..."
Abstract

Cited by 10 (7 self)
 Add to MetaCart
Abstract. Inference proceeds from ingredients chosen by the analyst and data. To validate any inferences drawn it is essential that the inputs chosen be deemed appropriate for the data. In the Bayesian context these inputs consist of both the sampling model and the prior. There are thus two possibilities for failure: the data may not have arisen from the sampling model, or the prior may place most of its mass on parameter values that are not feasible in light of the data (referred to here as priordata conflict). Failure of the sampling model can only be fixed by modifying the model, while priordata conflict can be overcome if sufficient data is available. We examine how to assess whether or not a priordata conflict exists, and how to assess when its effects can be ignored for inferences. The concept of priordata conflict is seen to lead to a partial characterization of what is meant by a noninformative prior or a noninformative sequence of priors.
Bayesian Curve Estimation By Polynomials of Random Order
, 1995
"... this paper is to provide a Bayesian (non or semiparametric) alternative to these approaches by making the order of the polynomial unknown and random (in a Bayesian sense) simultaneously with other model parameters. The resulting mixture distributions covers a rich class of models. Experience suggest ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
this paper is to provide a Bayesian (non or semiparametric) alternative to these approaches by making the order of the polynomial unknown and random (in a Bayesian sense) simultaneously with other model parameters. The resulting mixture distributions covers a rich class of models. Experience suggests that the orders of the polynomial acquiring substantial posterior probabilities are never very high and that mixing over low order polynomials can capture even quite complicated functions. Other advantages are that it is easy to differentiate, integrate and not too difficult to put constraints on the function. Of course, the global nature of polynomial fitting implies that there will be problems if the unknown curve is too wiggly and so we extend our methodology to polynomials of random order with change points and to piecewise polynomials of random order.
Was it a car or a cat I saw? An Analysis of Response Times for Word Recognition
"... We model the response times for word recognition collected in experimental trials conducted on four subjects. Because of the sequential nature of the experiment and the fact that several replications of similar trials were conducted on each subject, the assumption of i.i.d. response times (often enc ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
We model the response times for word recognition collected in experimental trials conducted on four subjects. Because of the sequential nature of the experiment and the fact that several replications of similar trials were conducted on each subject, the assumption of i.i.d. response times (often encountered in the psychology literature) is untenable. We consider Bayesian hierarchical models in which the response times are described as conditionally i.i.d. Weibull random variables given the parameters of the Weibull distribution. The sequential dependencies, as well as the effects of response accuracy, word characteristics, and subject specific learning processes are incorporated via a linear regression model for the logarithm of the scale parameter of the Weibull distribution. We compare the inferences from our analysis with those obtained by means of instruments that are commonly used in the cognitive psychology arena. We pay close attention to the quality of the fits, the adequacy of the assumptions, and their impact on the inferential conclusions.
Bayesian checking of hierarchical models
, 2004
"... Hierarchical models are increasingly used in many applications. Along with this increase use comes a desire to investigate whether the model is compatible with the observed data. Bayesian methods are well suited to eliminate the many (nuisance) parameters in these complicated models; in this paper w ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
Hierarchical models are increasingly used in many applications. Along with this increase use comes a desire to investigate whether the model is compatible with the observed data. Bayesian methods are well suited to eliminate the many (nuisance) parameters in these complicated models; in this paper we investigate Bayesian methods for model checking. Since we contemplate model checking as a preliminary, exploratory analysis, we concentrate in objective Bayesian methods in which careful specification of an informative prior distribution is avoided. Numerous examples are given and different proposals are investigated. Key words and phrases: model checking; model criticism; objective Bayesian methods; pvalues. 1
Measures of Surprise in Bayesian Analysis
 Duke University
, 1997
"... Measures of surprise refer to quantifications of the degree of incompatibility of data with some hypothesized model H 0 without any reference to alternative models. Traditional measures of surprise have been the pvalues, which are however known to grossly overestimate the evidence against H 0 . Str ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Measures of surprise refer to quantifications of the degree of incompatibility of data with some hypothesized model H 0 without any reference to alternative models. Traditional measures of surprise have been the pvalues, which are however known to grossly overestimate the evidence against H 0 . Strict Bayesian analysis calls for an explicit specification of all possible alternatives to H 0 so Bayesians have not made routine use of measures of surprise. In this report we CRITICALLY REVIEw the proposals that have been made in this regard. We propose new modifications, stress the connections with robust Bayesian analysis and discuss the choice of suitable predictive distributions which allow surprise measures to play their intended role in the presence of nuisance parameters. We recommend either the use of appropriate likelihoodratio type measures or else the careful calibration of pvalues so that they are closer to Bayesian answers. Key words and phrases. Bayes factors; Bayesian pvalues; Bayesian robustness; Conditioning; Model checking; Predictive distributions. 1.
A Comparison Between PValues for Goodnessoffit Checking
, 2001
"... The problem of checking the compatibility of a proposed parametric model with the observed data is an old one. If no alternative models are proposed, Bayes factors are precluded and only measures of `surprise' can be given. The value is, by far, the most ubiquitous of such measures. Also, model che ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
The problem of checking the compatibility of a proposed parametric model with the observed data is an old one. If no alternative models are proposed, Bayes factors are precluded and only measures of `surprise' can be given. The value is, by far, the most ubiquitous of such measures. Also, model checking is often done in a `casual' manner using statistics which are not distributionfree nor even parameterfree under the assumed model. In these situations, we compare several values that can be used for model checking. We investigate both their distributions under the `null' model, and their power under families of alternatives. Keywords: MODEL CHECKING; NULL DISTRIBUTION OF VALUES; PARTIAL POSTERIOR PREDICTIVE VALUE; PLUGIN VALUE; POSTERIOR PREDICTIVE VALUE; POWER COMPARISON. 1.
Multivariate Regression Analysis of Panel Data with Binary Outcomes applied to Unemployment Data
"... this paper we present an application of this algorithm to unemployment data from the Panel Study of Income Dynamics involving 11 waves of the panel study. In addition we adapt Bayesian model checking techniques based on the posterior predictive distribution (see for example Gelman et al. (1996)) for ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
this paper we present an application of this algorithm to unemployment data from the Panel Study of Income Dynamics involving 11 waves of the panel study. In addition we adapt Bayesian model checking techniques based on the posterior predictive distribution (see for example Gelman et al. (1996)) for the multivariate probit model. These help to identify mean and correlation specification which fit the data well.
Nonparametric GoodnessofFit
"... This paper develops an approach to testing the adequacy of both classical and Bayesian models given univariate sample data. An important feature of the approach is that we are able to test the practical scientific hypothesis of whether the true underlying model is close to some hypothesized model. T ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
This paper develops an approach to testing the adequacy of both classical and Bayesian models given univariate sample data. An important feature of the approach is that we are able to test the practical scientific hypothesis of whether the true underlying model is close to some hypothesized model. The notion of closeness is based on measurement precision and requires the introduction of a metric for which we consider the Kolmogorov distance. The approach is nonparametric in the sense that the alternative model is a Dirichlet process. Keywords : Monte Carlo, hypothesis testing, nonparametric Bayesian inference, prior elicitation. 1. INTRODUCTION Although Bayesian applications have seen unprecedented growth in the last 10 years, there is no consensus on the correct approach to Bayesian model checking. A selection of diverse approaches that address Bayesian model checking includes Guttman (1967), Guttman, Dutter and Freeman (1978), Chaloner and Brant (1988), Weiss (1994), Gelman, Meng ...