Results 1  10
of
108
Implementing approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations: A manual for the inlaprogram
, 2008
"... Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalised) linear models, (generalised) additive models, smoothingspline models, statespace models, semiparametric regression, spatial and spatiotemp ..."
Abstract

Cited by 293 (20 self)
 Add to MetaCart
Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalised) linear models, (generalised) additive models, smoothingspline models, statespace models, semiparametric regression, spatial and spatiotemporal models, logGaussian Coxprocesses, geostatistical and geoadditive models. In this paper we consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with nonGaussian response variables. The posterior marginals are not available in closed form due to the nonGaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, both in terms of convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations
Bayesian PSplines
 Journal of Computational and Graphical Statistics
, 2004
"... Psplines are an attractive approach for modelling nonlinear smooth effects of covariates within the generalized additive and varying coefficient models framework. In this paper we propose a Bayesian version for Psplines and generalize the approach for one dimensional curves to two dimensional surf ..."
Abstract

Cited by 122 (26 self)
 Add to MetaCart
(Show Context)
Psplines are an attractive approach for modelling nonlinear smooth effects of covariates within the generalized additive and varying coefficient models framework. In this paper we propose a Bayesian version for Psplines and generalize the approach for one dimensional curves to two dimensional surface fitting for modelling interactions between metrical covariates. A Bayesian approach to Psplines has the advantage of allowing for simultaneous estimation of smooth functions and smoothing parameters. Moreover, it can easily be extended to more complex formulations, for example to mixed models with random effects for serially or spatially correlated response. Additionally, the assumption of constant smoothing parameters can be replaced by allowing the smoothing parameters to be locally adaptive. This is particularly useful in situations with changing curvature of the underlying smooth function or where the function is highly oscillating. Inference is fully Bayesian and uses recent MCMC techniques for drawing random samples from the posterior. In a couple of simulation studies the performance of Bayesian Psplines is studied and compared to other approaches in the literature. We illustrate the approach by a complex application on rents for flats in Munich.
An Explicit Link between Gaussian Fields and . . .
 PREPRINTS IN MATHEMATICAL SCIENCES
, 2010
"... Continuously indexed Gaussian fields (GFs) is the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitive interpretation of its properties. On the computational side, GFs are hampered with the bign problem, ..."
Abstract

Cited by 114 (17 self)
 Add to MetaCart
Continuously indexed Gaussian fields (GFs) is the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitive interpretation of its properties. On the computational side, GFs are hampered with the bign problem, since the cost of factorising dense matrices is cubic in the dimension. Although the computational power today is alltimehigh, this fact seems still to be a computational bottleneck in applications. Along with GFs, there is the class of Gaussian Markov random fields (GMRFs) which are discretely indexed. The Markov property makes the involved precision matrix sparse which enables the use of numerical algorithms for sparse matrices, that for fields in R 2 only use the squareroot of the time required by general algorithms. The specification of a GMRF is through its full conditional distributions but its marginal properties are not transparent in such a parametrisation. In this paper, we show that using an approximate stochastic weak solution to (linear) stochastic partial differential equations (SPDEs), we can, for some GFs in the Matérn class, provide an explicit link, for any triangulation of R d, between GFs and GMRFs. The consequence is that we can take the best from the two worlds and do the modelling using GFs but do the computations using GMRFs. Perhaps more importantly,
Penalized structured additive regression for spacetime data: a Bayesian perspective
 Statistica Sinica
, 2004
"... We propose extensions of penalized spline generalized additive models for analysing spacetime regression data and study them from a Bayesian perspective. Nonlinear effects of continuous covariates and time trends are modelled through Bayesian versions of penalized splines, while correlated spatia ..."
Abstract

Cited by 47 (21 self)
 Add to MetaCart
We propose extensions of penalized spline generalized additive models for analysing spacetime regression data and study them from a Bayesian perspective. Nonlinear effects of continuous covariates and time trends are modelled through Bayesian versions of penalized splines, while correlated spatial effects follow a Markov random field prior. This allows to treat all functions and effects within a unified general framework by assigning appropriate priors with different forms and degrees of smoothness. Inference can be performed either with full (FB) or empirical Bayes (EB) posterior analysis. FB inference using MCMC techniques is a slight extension of own previous work. For EB inference, a computationally efficient solution is developed on the basis of a generalized linear mixed model representation. The second approach can be viewed as posterior mode estimation and is closely related to penalized likelihood estimation in a frequentist setting. Variance components, corresponding to smoothing parameters, are then estimated by using marginal likelihood. We carefully compare both inferential procedures in simulation studies and illustrate them through real data applications. The methodology is available in the open domain statistical package BayesX and as an Splus/R function.
Lang S: Generalized structured additive regression based on Bayesian P splines
 Computational Statistics & Data Analysis
"... Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM’s and extensions to generalized structured additive regression based on one or two dimensional Psplines as th ..."
Abstract

Cited by 45 (9 self)
 Add to MetaCart
Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM’s and extensions to generalized structured additive regression based on one or two dimensional Psplines as the main building block. The approach extends previous work by Lang and Brezger (2003) for Gaussian responses. Inference relies on Markov chain Monte Carlo (MCMC) simulation techniques, and is either based on iteratively weighted least squares (IWLS) proposals or on latent utility representations of (multi)categorical regression models. Our approach covers the most common univariate response distributions, e.g. the Binomial, Poisson or Gamma distribution, as well as multicategorical responses. As we will demonstrate through two applications on the forest health status of trees and a spacetime analysis of health insurance data, the approach allows realistic modelling of complex problems. We consider the enormous flexibility and extendability of our approach as a main advantage of Bayesian inference based on MCMC techniques compared to more traditional approaches. Software for the methodology presented in the paper is provided within the public domain package BayesX. Key words: geoadditive models, IWLS proposals, multicategorical response, structured additive
Approximate Bayesian inference for hierarchical Gaussian Markov random fields models
 Journal of Statistical Planning and Inference
, 2007
"... Many commonly used models in statistics can be formulated as (Bayesian) hierarchical Gaussian Markov random field models. These are characterised by assuming a (often large) Gaussian Markov random field (GMRF) as the second stage in the hierarchical structure and a few hyperparameters at the third s ..."
Abstract

Cited by 40 (8 self)
 Add to MetaCart
(Show Context)
Many commonly used models in statistics can be formulated as (Bayesian) hierarchical Gaussian Markov random field models. These are characterised by assuming a (often large) Gaussian Markov random field (GMRF) as the second stage in the hierarchical structure and a few hyperparameters at the third stage. Markov chain Monte Carlo is the common approach for Bayesian inference in such models. The variance of the Monte Carlo estimates is Op(M −1/2) where M is the number of samples in the chain so, in order to obtain precise estimates of marginal densities, say, we need M to be very large. Inspired by the fact that often oneblock and independence samplers can be constructed for hierarchical GMRF models, we will in this work investigate whether MCMC is really needed to estimate marginal densities, which often is the goal of the analysis. By making use of GMRFapproximations, we show by typical examples that marginal densities can indeed be very precisely estimated by deterministic schemes. The methodological and practical consequence of these findings are indeed positive. We conjecture that for many hierarchical GMRFmodels there is really no need for MCMC based inference to estimate marginal densities. Further, by making use of numerical methods for sparse matrices the computational costs of these deterministic schemes are nearly instant compared to the MCMC alternative. In particular, we discuss in detail the issue of computing marginal variances for GMRFs.
Double Markov Random Fields and Bayesian Image Segmentation
, 2002
"... Markov random fields are used extensively in modelbased approaches to image segmentation and, under the Bayesian paradigm, are implemented through Markov chain Monte Carlo (MCMC) methods. In this paper, we describe a class of such models (the double Markov random field) for images composed of severa ..."
Abstract

Cited by 28 (0 self)
 Add to MetaCart
(Show Context)
Markov random fields are used extensively in modelbased approaches to image segmentation and, under the Bayesian paradigm, are implemented through Markov chain Monte Carlo (MCMC) methods. In this paper, we describe a class of such models (the double Markov random field) for images composed of several textures, which we consider to be the natural hierarchical model for such a task. We show how several of the Bayesian approaches in the literature can be viewed as modifications of this model, made in order to make MCMC implementation possible. From a simulation study, conclusions are made concerning the performance of these modified models.
Approximating Hidden Gaussian Markov Random Fields
 JOURNAL OF THE ROYAL STATISTICAL SOCIETY, SERIES B
, 2003
"... This paper discusses how to construct approximations to a unimodal hidden Gaussian Markov random field on a graph of dimension n when the likelihood consists of mutually independent data. We demonstrate that a class of nonGaussian approximations can be constructed for a wide range of likelihood ..."
Abstract

Cited by 26 (4 self)
 Add to MetaCart
This paper discusses how to construct approximations to a unimodal hidden Gaussian Markov random field on a graph of dimension n when the likelihood consists of mutually independent data. We demonstrate that a class of nonGaussian approximations can be constructed for a wide range of likelihood models. They have the appealing properties that exact samples can be drawn from them, the normalisation constant is computable, and the computational complexity is only O(n 2 ) in the spatial case. The nonGaussian approximations are refined versions of a Gaussian approximation. The latter serves well if the likelihood is nearGaussian, but it is not sufficiently accurate when the likelihood is not nearGaussian or if n is large. The accuracy of our approximations can be tuned by intuitive parameters to near any precision. We apply
Markov random field models for highdimensional parameters in simulations of fluid flow in porous media
 Technometrics
, 2002
"... We give an approach for using flow information from a system of wells to characterize hydrologic properties of an aquifer. In particular, we consider experiments where an impulse of tracer fluid is injected along with the water at the input wells and its concentration is recorded over time at the up ..."
Abstract

Cited by 24 (8 self)
 Add to MetaCart
We give an approach for using flow information from a system of wells to characterize hydrologic properties of an aquifer. In particular, we consider experiments where an impulse of tracer fluid is injected along with the water at the input wells and its concentration is recorded over time at the uptake wells. We focus on characterizing the spatially varying permeability field which is a key attribute of the aquifer for determining flow paths and rates for a given flow experiment. As is standard for estimation from such flow data, we make use of complicated subsurface flow code which simulates the fluid flow through the aquifer for a particular well configuration and aquifer specification, which includes the permeability field over a grid. This illposed problem requires that some regularity be imposed on the permeability field. Typically this is accomplished by specifying a stationary Gaussian process model for the permeability field. Here we use an intrinsically stationary Markov random field which compares favorably and offers some additional flexibility and computational advantages. Our interest in quantifying uncertainty leads us to take a Bayesian approach, using Markov chain Monte Carlo for exploring the highdimensional posterior distribution. We demonstrate our approach with several examples.
Gaussian sampling by local perturbations
"... We present a technique for exact simulation of Gaussian Markov random fields (GMRFs), which can be interpreted as locally injecting noise to each Gaussian factor independently, followed by computing the mean/mode of the perturbed GMRF. Coupled with standard iterative techniques for the solution of s ..."
Abstract

Cited by 18 (1 self)
 Add to MetaCart
(Show Context)
We present a technique for exact simulation of Gaussian Markov random fields (GMRFs), which can be interpreted as locally injecting noise to each Gaussian factor independently, followed by computing the mean/mode of the perturbed GMRF. Coupled with standard iterative techniques for the solution of symmetric positive definite systems, this yields a very efficient sampling algorithm with essentially linear complexity in terms of speed and memory requirements, well suited to extremely large scale probabilistic models. Apart from synthesizing data under a Gaussian model, the proposed technique directly leads to an efficient unbiased estimator of marginal variances. Beyond Gaussian models, the proposed algorithm is also very useful for handling highly nonGaussian continuouslyvalued MRFs such as those arising in statistical image modeling or in the first layer of deep belief networks describing realvalued data, where the nonquadratic potentials coupling different sites can be represented as finite or infinite mixtures of Gaussians with the help of local or distributed latent mixture assignment variables. The Bayesian treatment of such models most naturally involves a block Gibbs sampler which alternately draws samples of the conditionally independent latent mixture assignments and the conditionally multivariate Gaussian continuous vector and we show that it can directly benefit from the proposed methods. 1