Abstract-In the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. We examine the relative entropy distance D,, between the true density and the Bayesian density and show that the asymptotic distance is (d/2Xlogn)+ c, where d is the dimension of the parameter vector. Therefore, the relative entropy rate D,,/n converges to zero at rate (logn)/n. The constant c, which we explicitly identify, depends only on the prior density function and the Fisher information matrix evaluated at the true parameter value. Consequences are given for density estima-tion, universal data compression, composite hypothesis testing, and stock-market portfolio selection. 1.

Particle filter methods are complex inference procedures, which combine importance sampling and Monte Carlo schemes, in order to consistently explore a sequence of multiple distributions of interest. The purpose of this article is to show that such methods can also offer an efficient estimation tool in "static" setups; in this case, &pi;(&theta;|y_1, ..., y_N) is the only posterior distribution of interest but the preliminary exploration of partial posteriors &pi;(&theta;|y_1, ..., y_N) (n < N) makes computing time savings possible. A complete "black-box" algorithm is proposed for independent or Markov models. Our method is shown to possibly challenge other common estimation procedures, in terms of robustness and execution time, especially when the sample size is important. Two classes of examples are discussed and illustrated by numerical results: mixture models and discrete generalized linear models.

The ¨generic viewpointässumption states that an observer is not in a special position relative to the scene. It is commonly used to disqualify scene interpretations that assume special viewpoints, following a binary decision that the viewpoint was either generic or accidental. In this paper, we apply Bayesian statistics to quantify the probability of a view, and so derive a useful tool to estimate scene parameters. This approach may increase the scope and accuracy of scene estimates. It applies to a range of vision problems. We show shape from shading examples, where we rank shapes or reflectance functions in cases where these are otherwise unknown. The rankings agree with the perceived values.

this paper, we study the behaviour of the posterior distribution as the sample size n tends to infinity where the dimension of the parameter space p = p n is also allowed to grow to infinity with n. This problem is of significant practical importance since in data analysis, one often uses a delicate model (i.e., with a large number of parameters) if one has enough data. In other words, one allows the dimension of the parameter to grow with the sample size. Moreover, nonparametric models can be approximated by parametric models with increasing dimension as discussed by Shibata (1981) and Diaconis and Freedman (1993). The frequentist version of this problem, namely consistency and asymptotic normality of M-estimates has been studied by Huber (1973), Yohai and Maronna (1979), Ringland (1983) and Portnoy (1984, 1985, 1986). In this paper we show that, under certain growth restrictions on the dimension depending on the design variables, the posterior distributions concentrate in the neighbourhoods of the true value of the parameter and admit a normal approximation. It seems that the present paper is the first attempt to study Bayesian asymptotic properties in models of increasing dimension. We observe that the condition required on the growth rate of the dimension p n is more stringent than its frequentist counterparts. Though no claim is made about the necessity of this condition on the growth of p n , we believe that there are at least three reasons to expect some difficulties if p n grows very fast with n. First, there is a long tail area which may substantially contribute to the posterior probabilities although the likelihood is small there. Secondly, our choice of the L

Let X k be a sequence of iid random variables taking values in a finite set, and consider the problem of estimating the law of X 1 in a Bayesian framework. We prove that the sequence of posterior distributions satisfies a large deviation principle, and give an explicit expression for the rate function. As an application, we obtain an asymptotic formula for the predictive probability of ruin in the classical gambler's ruin problem. 1 Introduction and preliminaries Let X be a Hausdorff topological space with Borel oe-algebra B, and let ¯ n be a sequence of probability measures on (X ; B). A rate function is a nonnegative lower semicontinuous function on X . We say that the sequence ¯ n satisfies the large deviation principle (LDP) with rate function I, if for all B 2 B, \Gamma inf x2B ffi I(x) lim inf n 1 n log ¯ n (B) lim sup n 1 n log ¯ n (B) \Gamma inf x2 ¯ B I(x): Here B ffi and ¯ B denote the interior and closure of B, respectively. Let\Omega be a finite set and ...

We address the classic problem of interval estimation of a binomial proportion. The Wald interval ^ p z =2 n 1=2 (^p(1 ^ p)) 1=2 is currently in near universal use. We first show that the coverage properties of the Wald interval are persistently poor and defy virtually all conventional wisdom. We then proceed to a theoretical comparison of the standard interval and four additional alternative intervals by asymptotic expansions of their coverage probabilities and expected lengths. The four additional interval methods we study in detail are the score-test interval (Wilson (1927)) the likelihood-ratio-test interval, a Jeffreys prior Bayesian interval and an interval suggested in Agresti and Coull (1998). The asymptotic expansions for coverage show that the first three of these alternative methods have coverages that uctuate about the nominal value, while the Agresti-Coull interval has a somewhat larger and more nearly conservative coverage function. For the five interval methods we al...

Laplace's method, a family of asymptotic methods used to approximate integrals, is presented as a potential candidate for the tool box of techniques used for knowledge acquisition and probabilistic inference in belief networks with continuous variables. This technique approximates posterior moments and marginal posterior distributions with reasonable accuracy [errors are O(n,2) for posterior means] in many interesting cases. The method also seems promising for computing approximations for Bayes factors for use in the context of model selection, model uncertainty and mixtures of pdfs. The limitations, regularity conditions and computational di culties for the implementation of Laplace's method are comparable to those associated with the methods of maximum likelihood and posterior mode analysis. 1

We consider higher order frequentist inference for the parametric component of a semiparametric model based on sampling from the posterior profile distribution. The first order validity of this procedure established by Lee, Kosorok and Fine (2005) is extended to second order validity in the setting where the infinite dimensional nuisance parameter achieves the parametric rate. Specifically, we obtain higher order estimates of the maximum profile likelihood estimator and of the efficient Fisher information. Moreover, we prove that an exact frequentist confidence interval for the parametric component at level alpha can be estimated by the alpha level credible set from the profile sampler with an error of order OP (n −1). As far as we are aware, these results are the first higher order frequentist results obtained for semiparametric estimation. A fully Bayesian interpretation is established under a certain data dependent prior. The theory is verified for three specific examples.

Foundation under Grant SES 0920903 and the USC Faculty Development Award. Schorfheide gratefully

We address the classic problem of interval estimation of a binomial proportion. The Wald interval p\Sigmaz ff=2 n \Gamma1=2 (p(1\Gamma p)) 1=2 is currently in near universal use. We first show that the coverage properties of the Wald interval are persistently poor and defy virtually all conventional wisdom. We then proceed to a theoretical comparison of the standard interval and four additional alternative intervals by asymptotic expansions of their coverage probabilities and expected lengths. Fortunately, the asymptotic expansions are remarkably accurate at rather modest sample sizes, such as n = 40, or sometimes even n = 20. The expansions show that an interval suggested in Agresti and Coull (1998) dominates the score interval (Wilson (1927)), the Jeffreys prior Bayesian interval, and also the standard interval in coverage probability. However, the asymptotic expansions for expected lengths show that the Agresti-Coull interval is always the longest of these, and the Jeffreys pr...