Results 1  10
of
17
A unifying view of sparse approximate Gaussian process regression
 Journal of Machine Learning Research
, 2005
"... We provide a new unifying view, including all existing proper probabilistic sparse approximations for Gaussian process regression. Our approach relies on expressing the effective prior which the methods are using. This allows new insights to be gained, and highlights the relationship between existin ..."
Abstract

Cited by 83 (3 self)
 Add to MetaCart
We provide a new unifying view, including all existing proper probabilistic sparse approximations for Gaussian process regression. Our approach relies on expressing the effective prior which the methods are using. This allows new insights to be gained, and highlights the relationship between existing methods. It also allows for a clear theoretically justified ranking of the closeness of the known approximations to the corresponding full GPs. Finally we point directly to designs of new better sparse approximations, combining the best of the existing strategies, within attractive computational constraints.
Variational learning of inducing variables in sparse Gaussian processes
 In Artificial Intelligence and Statistics 12
, 2009
"... Sparse Gaussian process methods that use inducing variables require the selection of the inducing inputs and the kernel hyperparameters. We introduce a variational formulation for sparse approximations that jointly infers the inducing inputs and the kernel hyperparameters by maximizing a lower bound ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
Sparse Gaussian process methods that use inducing variables require the selection of the inducing inputs and the kernel hyperparameters. We introduce a variational formulation for sparse approximations that jointly infers the inducing inputs and the kernel hyperparameters by maximizing a lower bound of the true log marginal likelihood. The key property of this formulation is that the inducing inputs are defined to be variational parameters which are selected by minimizing the KullbackLeibler divergence between the variational distribution and the exact posterior distribution over the latent function values. We apply this technique to regression and we compare it with other approaches in the literature. 1
Approximation Methods for Gaussian Process Regression
, 2007
"... A wealth of computationally efficient approximation methods for Gaussian process regression have been recently proposed. We give a unifying overview of sparse approximations, following QuiñoneroCandela and Rasmussen (2005), and a brief review of approximate matrixvector multiplication methods. 1 ..."
Abstract

Cited by 18 (3 self)
 Add to MetaCart
A wealth of computationally efficient approximation methods for Gaussian process regression have been recently proposed. We give a unifying overview of sparse approximations, following QuiñoneroCandela and Rasmussen (2005), and a brief review of approximate matrixvector multiplication methods. 1
Dimensionality Estimation, Manifold Learning and Function Approximation using Tensor Voting
, 2010
"... We address instancebased learning from a perceptual organization standpoint and present methods for dimensionality estimation, manifold learning and function approximation. Under our approach, manifolds in highdimensional spaces are inferred by estimating geometric relationships among the input in ..."
Abstract

Cited by 6 (5 self)
 Add to MetaCart
We address instancebased learning from a perceptual organization standpoint and present methods for dimensionality estimation, manifold learning and function approximation. Under our approach, manifolds in highdimensional spaces are inferred by estimating geometric relationships among the input instances. Unlike conventional manifold learning, we do not perform dimensionality reduction, but instead perform all operations in the original input space. For this purpose we employ a novel formulation of tensor voting, which allows an ND implementation. Tensor voting is a perceptual organization framework that has mostly been applied to computer vision problems. Analyzing the estimated local structure at the inputs, we are able to obtain reliable dimensionality estimates at each instance, instead of a global estimate for the entire data set. Moreover, these local dimensionality and structure estimates enable us to measure geodesic distances and perform nonlinear interpolation for data sets with varying density, outliers, perturbation and intersections, that cannot be handled by stateoftheart methods. Quantitative results on the estimation of local manifold structure using ground truth data are presented. In addition, we compare our approach with several leading methods for manifold learning at the task of measuring geodesic distances. Finally, we show competitive function approximation results on real data.
SemiSupervised Training of Models for AppearanceBased Statistical Object Detection Methods
, 2004
"... Appearancebased object detection systems using statistical models have proven quite successful. They can reliably detect textured, rigid objects in a variety of poses, lighting conditions and scales. However, the construction of these systems is timeconsuming and difficult because a large number o ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
Appearancebased object detection systems using statistical models have proven quite successful. They can reliably detect textured, rigid objects in a variety of poses, lighting conditions and scales. However, the construction of these systems is timeconsuming and difficult because a large number of training examples must be collected and manually labeled in order to capture variations in object appearance. Typically, this requires indicating which regions of the image correspond to the object to be detected, and which belong to background clutter, as well as marking key landmark locations on the object. The goal of this work is to pursue and evaluate approaches which reduce the amount of fully labeled examples needed, by training these models in a semisupervised manner. To this end, we develop approaches based on ExpectationMaximization and selftraining that utilize a small number of fully labeled training examples in combination with a set of "weakly labeled" examples. This is advantageous in that weakly labeled data are inherently less costly to generate, since the label information is specified in an uncertain or incomplete fashion. For example, a weakly labeled image might be labeled as containing the training object, with the object location and scale left unspecified. In this work we analyze the performance of the techniques developed through a comprehensive empirical investigation. We find that supplementing a small fully labeled training set with weakly labeled data in the training process reliably improves detector performance for a variety of detection approaches. The outcome is the identification of successful approaches and key issues that are central to achieving good performance in the semisupervised training of object detection systems.
How to choose the covariance for gaussian process regression independently of the basis
 In Proc. Gaussian Processes in Practice Workshop
, 2006
"... In Gaussian process regression, both the basis functions and their prior distribution are simultaneously specified by the choice of the covariance function. In certain problems one would like to choose the covariance independently of the basis functions (e. g., in polynomial signal processing or Wie ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
In Gaussian process regression, both the basis functions and their prior distribution are simultaneously specified by the choice of the covariance function. In certain problems one would like to choose the covariance independently of the basis functions (e. g., in polynomial signal processing or Wiener and Volterra analysis). We propose a solution to this problem that approximates the desired covariance function at a finite set of input points for arbitrary choices of basis functions. Our experiments show that this additional degree of freedom can lead to improved regression performance. 1.
Domain Decomposition Approach for Fast Gaussian Process Regression of Large Spatial Datasets
"... Editor: Gaussian process regression is a flexible and powerful tool for machine learning, but the high computational complexity hinders its broader applications. In this paper, we propose a new approach for fast computation of Gaussian process regression with a focus on large spatial datasets. The a ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Editor: Gaussian process regression is a flexible and powerful tool for machine learning, but the high computational complexity hinders its broader applications. In this paper, we propose a new approach for fast computation of Gaussian process regression with a focus on large spatial datasets. The approach decomposes the domain of a regression function into small subdomains and infers a local piece of the regression function for each subdomain. We explicitly address the mismatch problem of the local pieces on the boundaries of neighboring subdomains by imposing continuity constraints. The new approach has comparable or better computation complexity as other competing methods, but it is easier to be parallelized for faster computation. Moreover, the method can be adaptive to nonstationary features because of its local nature and, in particular, its use of different hyperparameters of the covariance function for different local regions. We illustrate application of the method and demonstrate its advantages over existing methods using two synthetic datasets and two real spatial datasets.
Analysis of some methods for reduced rank Gaussian process regression
 PROC. HAMILTON SUMMER SCHOOL ON SWITCHING AND LEARNING IN
, 2004
"... While there is strong motivation for using Gaussian Processes (GPs) due to their excellent performance in regression and classification problems, their computational complexity makes them impractical when the size of the training set exceeds a few thousand cases. This has motivated the recent proli ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
While there is strong motivation for using Gaussian Processes (GPs) due to their excellent performance in regression and classification problems, their computational complexity makes them impractical when the size of the training set exceeds a few thousand cases. This has motivated the recent proliferation of a number of costeffective approximations to GPs, both for classification and for regression. In this paper we analyze one popular approximation to GPs for regression: the reduced rank approximation. While generally GPs are equivalent to infinite linear models, we show that Reduced Rank Gaussian Processes (RRGPs) are equivalent to finite sparse linear models. We also introduce the concept of degenerate GPs and show that they correspond to inappropriate priors. We show how to modify the RRGP to prevent it from being degenerate at test time. Training RRGPs consists both in learning the covariance function hyperparameters and the support set. We propose a method for learning hyperparameters for a given support set. We also
Efficient implementation of the AIREML iteration for variance component QTL analysis
, 2007
"... Regions in the genome that affect complex traits, quantitative trait loci (QTL), can be identified using statistical analysis of genetic and phenotypic data. When restricted maximumlikelihood (REML) models are used, the mapping procedure is normally computationally demanding. We develop a new effic ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Regions in the genome that affect complex traits, quantitative trait loci (QTL), can be identified using statistical analysis of genetic and phenotypic data. When restricted maximumlikelihood (REML) models are used, the mapping procedure is normally computationally demanding. We develop a new efficient computational scheme for QTL mapping using variance component analysis and the AIREML algorithm. The algorithm uses an exact or approximative lowrank representation of the identitybydescent matrix, which combined with the Woodbury formula for matrix inversion results in that the computations in the AIREML iteration body can be performed more efficiently. For cases where an exact lowrank representation of the IBD matrix is available apriori, the improved AIREML algorithm normally runs almost twice as fast compared to the standard version. When an exact lowrank representation is not available, a truncated spectral decomposition is used to determine a lowrank approximation. We show that also in this case, the computational efficiency of the AIREML scheme can often be significantly improved.
Parallel geostatistics for sparse and dense datasets
"... Very large spatiallyreferenced datasets, for example, those derived from satellitebased sensors which sample across the globe or large monitoring networks of individual sensors, are becoming increasingly common and more widely available for use in environmental decision making. In large or dense s ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Very large spatiallyreferenced datasets, for example, those derived from satellitebased sensors which sample across the globe or large monitoring networks of individual sensors, are becoming increasingly common and more widely available for use in environmental decision making. In large or dense sensor networks, huge quantities of data can be collected over small time periods. In many applications the generation of maps, or predictions at specific locations, from the data in (near) realtime is crucial. Geostatistical operations such as interpolation are vital in this mapgeneration process and in emergency situations, the resulting predictions need to be available almost instantly, so that decision makers can make informed decisions and define risk and evacuation zones. It is also helpful when analysing data in less time critical applications, for example when interacting directly with the data for exploratory analysis, that the algorithms are responsive within a reasonable time frame.