Results 1  10
of
39
Task Clustering and Gating for Bayesian Multitask Learning
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2003
"... Modeling a collection of similar regression or classification tasks can be improved by making the tasks `learn from each other'. In machine learning, this subject is approached through `multitask learning', where parallel tasks are modeled as multiple outputs of the same network. In multilevel anal ..."
Abstract

Cited by 109 (1 self)
 Add to MetaCart
Modeling a collection of similar regression or classification tasks can be improved by making the tasks `learn from each other'. In machine learning, this subject is approached through `multitask learning', where parallel tasks are modeled as multiple outputs of the same network. In multilevel analysis this is generally implemented through the mixedeffects linear model where a distinction is made between `fixed effects', which are the same for all tasks, and `random effects', which may vary between tasks. In the present article we will adopt a Bayesian approach in which some of the model parameters are shared (the same for all tasks) and others more loosely connected through a joint prior distribution that can be learned from the data. We seek in this way to combine the best parts of both the statistical multilevel approach and the neural network machinery. The standard
Bayesian inference for generalized additive mixed models based on markov random field priors
 C
, 2001
"... Summary. Most regression problems in practice require ¯exible semiparametric forms of the predictor for modelling the dependence of responses on covariates. Moreover, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in lo ..."
Abstract

Cited by 64 (19 self)
 Add to MetaCart
Summary. Most regression problems in practice require ¯exible semiparametric forms of the predictor for modelling the dependence of responses on covariates. Moreover, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal or spatial data. We present a uni®ed approach for Bayesian inference via Markov chain Monte Carlo simulation in generalized additive and semiparametric mixed models. Different types of covariates, such as the usual covariates with ®xed effects, metrical covariates with nonlinear effects, unstructured random effects, trend and seasonal components in longitudinal data and spatial covariates, are all treated within the same general framework by assigning appropriate Markov random ®eld priors with different forms and degrees of smoothness. We applied the approach in several casestudies and consulting cases, showing that the methods are also computationally feasible in problems with many covariates and large data sets. In this paper, we choose two typical applications.
Nonparametric Function Estimation for Clustered Data When the Predictor is Measured Without/With Error
 Journal of the American Statistical Association
, 1999
"... We consider local polynomial kernel regression with a single covariate for clustered data using estimating equations. We assume that at most m < # observations are available on each cluster. In the case of random regressors, with no measurement error in the predictor, we show that it is generally ..."
Abstract

Cited by 31 (6 self)
 Add to MetaCart
We consider local polynomial kernel regression with a single covariate for clustered data using estimating equations. We assume that at most m < # observations are available on each cluster. In the case of random regressors, with no measurement error in the predictor, we show that it is generally the best strategy to ignore entirely the correlation structure within each cluster, and instead to pretend that all observations are independent. In the further special case of longitudinal data on individuals with fixed common observation times, we show that equivalent to the pooled data approach is the strategy of fitting separate nonparametric regressions at each observation time and constructing an optimal weighted average. We also consider what happens when the predictor is measured with error. Using the SIMEX approach to correct for measurement error, we construct an asymptotic theory for both the pooled and weighted average estimators. Surprisingly, for the same amount of smoothing, t...
Lang S: Generalized structured additive regression based on Bayesian P splines
 Computational Statistics & Data Analysis
"... Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM’s and extensions to generalized structured additive regression based on one or two dimensional Psplines as th ..."
Abstract

Cited by 27 (7 self)
 Add to MetaCart
Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM’s and extensions to generalized structured additive regression based on one or two dimensional Psplines as the main building block. The approach extends previous work by Lang and Brezger (2003) for Gaussian responses. Inference relies on Markov chain Monte Carlo (MCMC) simulation techniques, and is either based on iteratively weighted least squares (IWLS) proposals or on latent utility representations of (multi)categorical regression models. Our approach covers the most common univariate response distributions, e.g. the Binomial, Poisson or Gamma distribution, as well as multicategorical responses. As we will demonstrate through two applications on the forest health status of trees and a spacetime analysis of health insurance data, the approach allows realistic modelling of complex problems. We consider the enormous flexibility and extendability of our approach as a main advantage of Bayesian inference based on MCMC techniques compared to more traditional approaches. Software for the methodology presented in the paper is provided within the public domain package BayesX. Key words: geoadditive models, IWLS proposals, multicategorical response, structured additive
Penalized structured additive regression for spacetime data: a Bayesian perspective
 STATISTICA SINICA
, 2004
"... We propose extensions of penalized spline generalized additive models for analyzing spacetime regression data and study them from a Bayesian perspective. Nonlinear effects of continuous covariates and time trends are modelled through Bayesian versions of penalized splines, while correlated spati ..."
Abstract

Cited by 16 (11 self)
 Add to MetaCart
We propose extensions of penalized spline generalized additive models for analyzing spacetime regression data and study them from a Bayesian perspective. Nonlinear effects of continuous covariates and time trends are modelled through Bayesian versions of penalized splines, while correlated spatial effects follow a Markov random field prior. This allows to treat all functions and effects within a unified general framework by assigning appropriate priors with different forms and degrees of smoothness. Inference can be performed either with full (FB) or empirical Bayes (EB) posterior analysis. FB inference using MCMC techniques is a slight extension of previous work. For EB inference, a computationally efficient solution is developed on the basis of a generalized linear mixed model representation. The second approach can be viewed as posterior mode estimation and is closely related to penalized likelihood estimation in a frequentist setting. Variance components, corresponding to inverse smoothing parameters, are then estimated by marginal likelihood. We carefully compare both inferential procedures in simulation studies and illustrate them through data applications. The methodology is available in the open domain statistical package BayesX and as an Splus/R function.
Spatial models for line transect sampling
 Journal of Agricultural, Biological and Environmental Statistics
, 2004
"... This article develops methods for fitting spatial models to line transect data. These allow animal density to be related to topographical, environmental, habitat, and other spatial variables, helping wildlife managers to identify the factors that affect abundance. They also enable estimation of abun ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
This article develops methods for fitting spatial models to line transect data. These allow animal density to be related to topographical, environmental, habitat, and other spatial variables, helping wildlife managers to identify the factors that affect abundance. They also enable estimation of abundance for any subarea of interest within the surveyed region, and potentially yield estimates of abundance from sightings surveys for which the survey design could not be randomized, such as surveys conducted from platforms of opportunity. The methods are illustrated through analyses of data from a shipboard sightings survey of minke whales in the Antarctic.
Spline adaptation in extended linear models
 Statistical Science
, 2002
"... Abstract. In many statistical applications, nonparametric modeling can provide insight into the features of a dataset that are not obtainable by other means. One successful approach involves the use of (univariate or multivariate) spline spaces. As a class, these methods have inherited much from cla ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
Abstract. In many statistical applications, nonparametric modeling can provide insight into the features of a dataset that are not obtainable by other means. One successful approach involves the use of (univariate or multivariate) spline spaces. As a class, these methods have inherited much from classical tools for parametric modeling. For example, stepwise variable selection with spline basis terms is a simple scheme for locating knots (breakpoints) in regions where the data exhibit strong, local features. Similarly, candidate knot con gurations (generated by this or some other search technique), are routinely evaluated with traditional selection criteria like AIC or BIC. In short, strategies typically applied in parametric model selection have proved useful in constructing exible, lowdimensional models for nonparametric problems. Until recently, greedy, stepwise procedures were most frequently suggested in the literature. Researchinto Bayesian variable selection, however, has given rise to a number of new splinebased methods that primarily rely on some form of Markov chain Monte Carlo to identify promising knot locations. In this paper, we consider various alternatives to greedy, deterministic schemes, and present aBayesian framework for studying adaptation in the context of an extended linear model (ELM). Our major test cases are Logspline density estimation and (bivariate) Triogram regression models. We selected these because they illustrate a number of computational and methodological issues concerning model adaptation that arise in ELMs.
Comparing Curves Using Additive Models
 Journal of Quality Technology
, 2000
"... Advances in technology have increased dramatically the amount of data measured in industrial processes. Thousands of measurements are available nowadays in operations where previously only a single measurement, at a given point in time or space, was taken. These measurements allow the reconstruction ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
Advances in technology have increased dramatically the amount of data measured in industrial processes. Thousands of measurements are available nowadays in operations where previously only a single measurement, at a given point in time or space, was taken. These measurements allow the reconstruction of the whole profile or "signature" of the operation over time or space. Examples are the tonnage applied in a stamping press during a stroke and the density profile of particleboard. Many of these signatures have complicated forms that are not well modeled with parametric models. In this paper, a relatively new class of models called additive models is used to assess the sources of variation active on these signatures. The model contains a nonparametric or smooth portion to model the form of the signature, and a parametric portion to evaluate other sources of variation. An analysis of variance table is developed to test the magnitude of sources of variation. These techniques are illustrate...
Some asymptotic results on generalized penalized spline smoothing
 B
"... The paper discusses asymptotic properties of penalized spline smoothing if the spline basis increases with the sample size. The proof is provided in a generalized smoothing model allowing for nonnormal responses. The results are extended in two ways. First, assuming the spline coefficients to be a ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
The paper discusses asymptotic properties of penalized spline smoothing if the spline basis increases with the sample size. The proof is provided in a generalized smoothing model allowing for nonnormal responses. The results are extended in two ways. First, assuming the spline coefficients to be a priori normally distributed links the smoothing framework to generalized linear mixed models (GLMM). We consider the asymptotic rates such that Laplace approximation is justified and the resulting fits in the mixed model correspond to penalized spline estimates. Secondly, we make use of a fully Bayesian viewpoint by imposing a priori distribution on all parameters and coefficients. We argue that with the postulated rates at which the spline basis dimension increases with the sample size the posterior distribution of the spline coefficients is approximately normal. The validity of this result is investigated in finite samples by comparing Markov Chain Monte Carlo (MCMC) results with their asymptotic approximation in a simulation study. 1 1