Results 1  10
of
28
Bayesian Interpolation
 Neural Computation
, 1991
"... Although Bayesian analysis has been in use since Laplace, the Bayesian method of modelcomparison has only recently been developed in depth. In this paper, the Bayesian approach to regularisation and modelcomparison is demonstrated by studying the inference problem of interpolating noisy data. T ..."
Abstract

Cited by 522 (18 self)
 Add to MetaCart
Although Bayesian analysis has been in use since Laplace, the Bayesian method of modelcomparison has only recently been developed in depth. In this paper, the Bayesian approach to regularisation and modelcomparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other problems. Regularising constants are set by examining their posterior probability distribution. Alternative regularisers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. `Occam's razor' is automatically embodied by this framework. The way in which Bayes infers the values of regularising constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling. 1 Data modelling and Occam's razor In science, a central task is to develop and compare models to a...
Informationtheoretic asymptotics of Bayes methods
 IEEE Transactions on Information Theory
, 1990
"... AbstractIn the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. We examine the relative entropy distance D,, between the true density and the Bayesian densit ..."
Abstract

Cited by 107 (10 self)
 Add to MetaCart
AbstractIn the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. We examine the relative entropy distance D,, between the true density and the Bayesian density and show that the asymptotic distance is (d/2Xlogn)+ c, where d is the dimension of the parameter vector. Therefore, the relative entropy rate D,,/n converges to zero at rate (logn)/n. The constant c, which we explicitly identify, depends only on the prior density function and the Fisher information matrix evaluated at the true parameter value. Consequences are given for density estimation, universal data compression, composite hypothesis testing, and stockmarket portfolio selection. 1.
Bayesian analysis of DSGE models
 ECONOMETRICS REVIEW
, 2007
"... This paper reviews Bayesian methods that have been developed in recent years to estimate and evaluate dynamic stochastic general equilibrium (DSGE) models. We consider the estimation of linearized DSGE models, the evaluation of models based on Bayesian model checking, posterior odds comparisons, and ..."
Abstract

Cited by 53 (2 self)
 Add to MetaCart
This paper reviews Bayesian methods that have been developed in recent years to estimate and evaluate dynamic stochastic general equilibrium (DSGE) models. We consider the estimation of linearized DSGE models, the evaluation of models based on Bayesian model checking, posterior odds comparisons, and comparisons to vector autoregressions, as well as the nonlinear estimation based on a secondorder accurate model solution. These methods are applied to data generated from correctly specified and misspecified linearized DSGE models, and a DSGE model that was solved with a secondorder perturbation method. (JEL C11, C32, C51, C52)
Bayesian Neural Networks for Classification: How Useful is the Evidence Framework?
, 1998
"... This paper presents an empirical assessment of the Bayesian evidence framework for neural networks using four synthetic and four realworld classification problems. We focus on three issues; model selection, automatic relevance determination (ARD) and the use of committees. Model selection using the ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
This paper presents an empirical assessment of the Bayesian evidence framework for neural networks using four synthetic and four realworld classification problems. We focus on three issues; model selection, automatic relevance determination (ARD) and the use of committees. Model selection using the evidence criterion is only tenable if the number of training examples exceeds the number of network weights by a factor of five or ten. With this number of available examples, however, crossvalidation is a viable alternative. The ARD feature selection scheme is only useful in networks with many hidden units and for data sets containing many irrelevant variables. ARD is also useful as a hard feature selection method. Results on applying the evidence framework to the realworld data sets showed that committees of Bayesian networks achieved classification accuracies similar to the best alternative methods. Importantly, this was achievable with a minimum of human intervention. 1 Introduction ...
Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values
 ARTIFICIAL INTELLIGENCE
, 2004
"... We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes indefinitely larg ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes indefinitely large. Moreover, the variational posterior distribution is proved to be asymptotically normal.
Inadequacy of interval estimates corresponding to variational Bayesian approximations
 In Proc. 10th Int. Wrkshp Artificial Intelligence and Statistics (eds
, 2005
"... In this paper we investigate the properties of the covariance matrices associated with variational Bayesian approximations, based on data from mixture models, and compare them with the true covariance matrices, corresponding to Fisher information matrices. It is shown that the covariance matrices fr ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
In this paper we investigate the properties of the covariance matrices associated with variational Bayesian approximations, based on data from mixture models, and compare them with the true covariance matrices, corresponding to Fisher information matrices. It is shown that the covariance matrices from the variational Bayes approximations are normally ‘too small ’ compared with those for the maximum likelihood estimator, so that resulting interval estimates for the parameters will be unrealistically narrow, especially if the components of the mixture model are not well separated. 1
Asymptotic Normality of the Posterior in Relative Entropy
 IEEE Trans. Inform. Theory
, 1999
"... We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic compl ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
We show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case. The mode of convergence is in probability and in mean. Applications to codelengths in stochastic complexity and to sample size selection are briey discussed. Index Terms: Posterior density, asymptotic normality, relative entropy. Revision submitted to Trans. Inform Theory , 22 May 1998. This research was partially supported by NSERC Operating Grant 554891. The author is with the Department of Statistics, University of British Columbia, Room 333, 6356 Agricultural Road, Vancouver, BC, Canada V6T 1Z2. 1 I.
Easy Computation of Bayes Factors and Normalizing Constants for Mixture Models via Mixture Importance Sampling
, 2001
"... We propose a method for approximating integrated likelihoods, or posterior normalizing constants, in finite mixture models, for which analytic approximations such as the Laplace method are invalid. Integrated likelihoods are key components of Bayes factors and of the posterior model probabilities us ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
We propose a method for approximating integrated likelihoods, or posterior normalizing constants, in finite mixture models, for which analytic approximations such as the Laplace method are invalid. Integrated likelihoods are key components of Bayes factors and of the posterior model probabilities used in Bayesian model averaging. The method starts by formulating the model in terms of the unobserved group memberships, Z, and making these, rather than the model parameters, the variables of integration. The integral is then evaluated using importance sampling over the Z. The tricky part is choosing the importance sampling function, and we study the use of mixtures as importance sampling functions. We propose two forms of this: defensive mixture importance sampling (DMIS), and Zdistance importance sampling. We choose the parameters of the mixture adaptively, and we show how this can be done so as to approximately minimize the variance of the approximation to the integral.
Bayesian beamforming for DOA uncertainties: Theory and implementation
 IEEE Trans. Signal Processing
"... Abstract—A Bayesian approach to adaptive narrowband beamforming for uncertain source directionofarrival (DOA) is presented. The DOA is modeled as a random variable with prior statistics that describe its level of uncertainty. The Bayesian beamformer is the corresponding minimum meansquare error ( ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract—A Bayesian approach to adaptive narrowband beamforming for uncertain source directionofarrival (DOA) is presented. The DOA is modeled as a random variable with prior statistics that describe its level of uncertainty. The Bayesian beamformer is the corresponding minimum meansquare error (MMSE) estimator, which can be viewed as a mixture of directional beamformers combined according to the posterior distribution of the DOA given the data. Under a deterministic DOA, the meansquare error (MSE) of the Bayesian beamformer becomes as low as that of the directional beamformer equipped with the DOA candidate in the prior set that is the closest to the true DOA at exponential rate, where closeness is defined in the Kullback–Leibler sense. Two efficient algorithms using a uniform linear array (ULA) are presented. The first method utilizes the efficiency of the fast Fourier transform (FFT) to compute the posterior distribution on a large number of DOA candidates. The second method approximates the posterior distribution by a Gaussian distribution, which leads to a directional beamformer incorporated with a particular spreading matrix and an adjusted DOA. Numerical simulations show that the proposed beamformer outperforms other related blind or robust beamforming algorithms over a wide range of signaltonoise ratios (SNRs). Index Terms—Adaptive beamforming, Bayesian model, directionofarrival (DOA) uncertainty, minimum meansquare error (MMSE) estimation. I.
Variational Bayes estimation of mixing coefficients
, 2004
"... We investigate theoretically some properties of variational Bayes approximations based on estimating the mixing coefficients of known densities. We show that, with probability 1 as the sample size n grows large, the iterative algorithm for the variational Bayes approximation converges locally to the ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We investigate theoretically some properties of variational Bayes approximations based on estimating the mixing coefficients of known densities. We show that, with probability 1 as the sample size n grows large, the iterative algorithm for the variational Bayes approximation converges locally to the maximum likelihood estimator at the rate of O(1/n). Moreover, the variational posterior distribution for the parameters is shown to be asymptotically normal with the same mean but a different covariance matrix compared with those for the maximum likelihood estimator. Furthermore we prove that the covariance matrix from the variational Bayes approximation is `too small' compared with that for the MLE, so that resulting interval estimates for the parameters will be unrealistically narrow.