Results 1  10
of
41
COMPUTING f(A)b VIA LEAST SQUARES POLYNOMIAL APPROXIMATIONS
, 2009
"... Given a certain function f, various methods have been proposed in the past for addressing the important problem of computing the the matrixvector product f(A)b without explicitly computing the matrix f(A). Such methods were typically used to compute a specific function f, a common case being that ..."
Abstract

Cited by 15 (7 self)
 Add to MetaCart
Given a certain function f, various methods have been proposed in the past for addressing the important problem of computing the the matrixvector product f(A)b without explicitly computing the matrix f(A). Such methods were typically used to compute a specific function f, a common case being that of the exponential. This paper discusses a procedure based on least squares polynomials that can, in principle, be applied to any (continuous) function f. The idea is to start by approximating the function by a spline of a desired accuracy. Then, a particular definition of the function inner product is invoked that facilitates the computation of the least squares polynomial to this spline function. Since the function is approximated by a polynomial, the matrix A is referenced only through a matrixvector multiplication. In addition, the choice of the inner product makes it possible to avoid numerical integration. As an important application, we consider the case when f(t) = √ t and A is a sparse, symmetric positivedefinite matrix, which arises in sampling from a Gaussian process distribution. The covariance matrix of the distribution is defined by using a covariance function that has a compact support, at a very large number of sites that are on a regular or irregular grid. We derive error bounds and show extensive numerical results to illustrate the effectiveness of the proposed technique.
Covariance Estimation: The GLM and Regularization Perspectives
"... Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent highdimensional data environment where enforcing the positivedefinit ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent highdimensional data environment where enforcing the positivedefiniteness constraint could be computationally expensive. We provide a survey of the progress made in modeling covariance matrices from the perspectives of generalized linear models (GLM) or parsimony and use of covariates in low dimensions, regularization (shrinkage, sparsity) for highdimensional data, and the role of various matrix factorizations. A viable and emerging regressionbased setup which is suitable for both the GLM and the regularization approaches is to link a covariance matrix, its inverse or their factors to certain regression models and then solve the relevant (penalized) least squares problems. We point out several instances of this regressionbased setup in the literature. A notable case is in the Gaussian graphical models where linear regressions with LASSO penalty are used to estimate the neighborhood of one node at a time (Meinshausen and Bühlmann, 2006). Some advantages
Fixeddomain asymptotic properties of tapered maximum likelihood estimators
 Ann. Statist
, 2009
"... When the spatial sample size is extremely large, which occurs in many environmental and ecological studies, operations on the large covariance matrix are a numerical challenge. Covariance tapering is a technique to alleviate the numerical challenges. Under the assumption that data are collected alo ..."
Abstract

Cited by 14 (1 self)
 Add to MetaCart
(Show Context)
When the spatial sample size is extremely large, which occurs in many environmental and ecological studies, operations on the large covariance matrix are a numerical challenge. Covariance tapering is a technique to alleviate the numerical challenges. Under the assumption that data are collected along a line in a bounded region, we investigate how the tapering affects the asymptotic efficiency of the maximum likelihood estimator (MLE) for the microergodic parameter in the Matérn covariance function by establishing the fixeddomain asymptotic distribution of the exact MLE and that of the tapered MLE. Our results imply that, under some conditions on the taper, the tapered MLE is asymptotically as efficient as the true MLE for the microergodic parameter in the Matérn model.
Bayesian Modeling with Gaussian Processes using the GPstuff Toolbox
, 2014
"... Gaussian processes (GP) are powerful tools for probabilistic modeling purposes. They can be used to define prior distributions over latent functions in hierarchical Bayesian models. The prior over functions is defined implicitly by the mean and covariance function, which determine the smoothness and ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
Gaussian processes (GP) are powerful tools for probabilistic modeling purposes. They can be used to define prior distributions over latent functions in hierarchical Bayesian models. The prior over functions is defined implicitly by the mean and covariance function, which determine the smoothness and variability of the function. The inference can then be conducted directly in the function space by evaluating or approximating the posterior process. Despite their attractive theoretical properties GPs provide practical challenges in their implementation. GPstuff is a versatile collection of computational tools for GP models compatible with Linux and Windows MATLAB and Octave. It includes, among others, various inference methods, sparse approximations and tools for model assessment. In this work, we review these tools and demonstrate the use of GPstuff in several models.
Estimation and prediction in spatial models with block composite likelihoods
 Journal of Computational and Graphical Statistics (To Appear
, 2013
"... A block composite likelihood is developed for estimation and prediction in large spatial datasets. The composite likelihood is constructed from the joint densities of pairs of adjacent spatial blocks. This allows large datasets to be split into many smaller datasets, each of which can be evaluated s ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
A block composite likelihood is developed for estimation and prediction in large spatial datasets. The composite likelihood is constructed from the joint densities of pairs of adjacent spatial blocks. This allows large datasets to be split into many smaller datasets, each of which can be evaluated separately, and combined through a simple summation. Estimates for unknown parameters are obtained by maximizing the block composite likelihood function. In addition, a new method for optimal spatial prediction under the block composite likelihood is presented. Asymptotic variances for both parameter estimates and predictions are computed using Godambe sandwich matrices. The approach gives considerable improvements in computational efficiency, and the composite structure obviates the need to load entire datasets into memory at once, completely avoiding memory limitations imposed by massive datasets. Moreover, computing time can be reduced even further by distributing the operations using parallel computing. A simulation study shows that composite likelihood estimates and predictions, as well as their corresponding asymptotic confidence intervals, are competitive with those based on the full likelihood. The procedure is demonstrated on one dataset from the mining industry and one dataset of satellite retrievals. The realdata examples
ACCURATE EMULATORS FOR LARGESCALE COMPUTER EXPERIMENTS
, 1203
"... Largescale computer experiments are becoming increasingly important in science. A multistep procedure is introduced to statisticians for modeling such experiments, which builds an accurate interpolator inmultiple steps. Inpractice, the procedureshows substantial improvements in overall accuracy, b ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
(Show Context)
Largescale computer experiments are becoming increasingly important in science. A multistep procedure is introduced to statisticians for modeling such experiments, which builds an accurate interpolator inmultiple steps. Inpractice, the procedureshows substantial improvements in overall accuracy, but its theoretical properties are not well established. We introduce the terms nominal and numeric error and decompose the overall error of an interpolator into nominal and numeric portions. Bounds on the numeric and nominal error are developed to show theoretically that substantial gains in overall accuracy can be attained with the multistep approach.
Approximate Bayesian Inference for Large Spatial Datasets Using Predictive Process Models
, 2010
"... This article addresses the challenges of estimating hierarchical spatial models to large datasets. With the increasing availability of geocoded scientific data, hierarchical models involving spatial processes have become a popular method for carrying out spatial inference. Such models are customar ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
This article addresses the challenges of estimating hierarchical spatial models to large datasets. With the increasing availability of geocoded scientific data, hierarchical models involving spatial processes have become a popular method for carrying out spatial inference. Such models are customarily estimated using Markov chain Monte Carlo algorithms that, while immensely flexible, can become prohibitively expensive. In particular, fitting hierarchical spatial models often involves expensive decompositions of dense matrices whose computational complexity increases in cubic order with the number of spatial locations. Such matrix computations are required in each iteration of the Markov chain Monte Carlo algorithm, rendering them infeasible for large spatial data sets. This article proposes to address the computational challenges in modeling large spatial datasets by merging two recent developments. First, we use the predictive process model as a reducedrank spatial process, to diminish the dimensionality of the model. Then we proceed to develop a computational framework for estimating predictive process models using the integrated nested Laplace approximation. We discuss settings where the first stage likelihood
ON THE CONSISTENT SEPARATION OF SCALE AND VARIANCE FOR GAUSSIAN RANDOM FIELDS
, 906
"... We present fixed domain asymptotic results that establish consistent estimates of the variance and scale parameters for a Gaussian random field with a geometric anisotropic Matérn autocovariance in dimension d> 4. When d < 4 this is impossible due to the mutual absolute continuity of Matérn Ga ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
We present fixed domain asymptotic results that establish consistent estimates of the variance and scale parameters for a Gaussian random field with a geometric anisotropic Matérn autocovariance in dimension d> 4. When d < 4 this is impossible due to the mutual absolute continuity of Matérn Gaussian random fields with different scale and variance (see Zhang [33]). Informally, when d> 4, we show that one can estimate the coefficient on the principle irregular term accurately enough to get a consistent estimate of the coefficient on the second irregular term. These two coefficients can then be used to separate the scale and variance. We extend our results to the general problem of estimating a variance and geometric anisotropy for more general autocovariance functions. Our results illustrate the interaction between the accuracy of estimation, the smoothness of the random field, the dimension of the observation space, and the number of increments used for estimation. As a corollary, our results establish the orthogonality of Matérn Gaussian random fields with different parameters when d> 4. The case d = 4 is still open. 1. Introduction. A
Efficient Emulators of Computer Experiments Using Compactly Supported Correlation Functions, With An Application to Cosmology
 Annals of Applied Statistics
, 2011
"... ar ..."
1 Scaling Multidimensional Inference for Structured Gaussian Processes
, 1209
"... Abstract—Exact Gaussian Process (GP) regression has O(N 3) runtime for data size N, making it intractable for large N. Many algorithms for improving GP scaling approximate the covariance with lower rank matrices. Other work has exploited structure inherent in particular covariance functions, includi ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
Abstract—Exact Gaussian Process (GP) regression has O(N 3) runtime for data size N, making it intractable for large N. Many algorithms for improving GP scaling approximate the covariance with lower rank matrices. Other work has exploited structure inherent in particular covariance functions, including GPs with implied Markov structure, and equispaced inputs (both enable O(N) runtime). However, these GP advances have not been extended to the multidimensional input setting, despite the preponderance of multidimensional applications. This paper introduces and tests novel extensions of structured GPs to multidimensional inputs. We present new methods for additive GPs, showing a novel connection between the classic backfitting method and the Bayesian framework. To achieve optimal accuracycomplexity tradeoff, we extend this model with a novel variant of projection pursuit regression. Our primary result – projection pursuit Gaussian Process Regression – shows orders of magnitude speedup while preserving high accuracy. The natural second and third steps include nonGaussian observations and higher dimensional equispaced grid methods. We introduce novel techniques to address both of these necessary directions. We thoroughly illustrate the power of these three advances on several datasets, achieving close performance to the naive Full GP at orders of magnitude less cost. Index Terms—Gaussian Processes, Backfitting, ProjectionPursuit Regression, Kronecker matrices. 1