Results 1  10
of
38
Selecting the Number of Knots For Penalized Splines
, 2000
"... Penalized splines, or Psplines, are regression splines fit by leastsquares with a roughness penaly. Psplines have much in common with smoothing splines, but the type of penalty used with a Pspline is somewhat more general than for a smoothing spline. Also, the number and location of the knots ..."
Abstract

Cited by 44 (7 self)
 Add to MetaCart
Penalized splines, or Psplines, are regression splines fit by leastsquares with a roughness penaly. Psplines have much in common with smoothing splines, but the type of penalty used with a Pspline is somewhat more general than for a smoothing spline. Also, the number and location of the knots of a Pspline is not fixed as with a smoothing spline. Generally, the knots of a Pspline are at fixed quantiles of the independent variable and the only tuning parameter to choose is the number of knots. In this article, the effects of the number of knots on the performance of Psplines are studied. Two algorithms are proposed for the automatic selection of the number of knots. The myoptic algorithm stops when no improvement in the generalized cross validation statistic (GCV) is noticed with the last increase in the number of knots. The full search examines all candidates in a fixed sequence of possible numbers of knots and chooses the candidate that minimizes GCV. The myoptic algo...
Geoadditive Models
, 2000
"... this paper is a recent article on modelbased geostatistics by Diggle, Tawn and Moyeed (1998) where pure kriging (i.e. no covariates) is the focus. Our paper inherits some of its aspects: modelbased and with mixed model connections. In particular the comment by Bowman (1998) in the ensuing discussi ..."
Abstract

Cited by 37 (1 self)
 Add to MetaCart
this paper is a recent article on modelbased geostatistics by Diggle, Tawn and Moyeed (1998) where pure kriging (i.e. no covariates) is the focus. Our paper inherits some of its aspects: modelbased and with mixed model connections. In particular the comment by Bowman (1998) in the ensuing discussion suggested that additive modelling would be a worthwhile extension. This paper essentially follows this suggestion. However, this paper is not the first to combine the notions of geostatistics and additive modelling. References known to us are Kelsall and Diggle (1998), Durban Reguera (1998) and Durban, Hackett, Currie and Newton (2000). Nevertheless, we believe that our approach has a number of attractive features (see (1)(4) above), not all shared by these references. Section 2 describes the motivating application and data in detail. Section 3 shows how one can express additive models as a mixed model, while Section 4 does the same for kriging and merges the two into the geoadditive model. Issues concerning the amount of smoothing are discussed in Section 5 and inferential aspects are treated in Section 6. Our analysis of the Upper Cape Cod reproductive data is presented in Section 7. Section 8 discusses extension to the generalised context.We close the paper with some disussion in Section 9. 2 Description of the application and data
Component Selection and Smoothing in Smoothing Spline Analysis of Variance Models
 COSSO. INSTITUTE OF STATISTICS MIMEO SERIES 2556, NCSU
, 2003
"... We propose a new method for model selection and model fitting in nonparametric regression models, in the framework of smoothing spline ANOVA. The "COSSO" is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in t ..."
Abstract

Cited by 29 (9 self)
 Add to MetaCart
We propose a new method for model selection and model fitting in nonparametric regression models, in the framework of smoothing spline ANOVA. The "COSSO" is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO applies a novel soft thresholding type operation to the function components and selects the correct model structure with probability tending to one. We give
Variable Selection and Model Building via Likelihood Basis Pursuit
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
, 2002
"... This paper presents a nonparametric penalized likelihood approach for variable selection and model building, called likelihood basis pursuit (LBP). In the setting of a tensor product reproducing kernel Hilbert space, we decompose the log likelihood into the sum of different functional components suc ..."
Abstract

Cited by 23 (10 self)
 Add to MetaCart
This paper presents a nonparametric penalized likelihood approach for variable selection and model building, called likelihood basis pursuit (LBP). In the setting of a tensor product reproducing kernel Hilbert space, we decompose the log likelihood into the sum of different functional components such as main effects and interactions, with each component represented by appropriate basis functions. The basis functions are chosen to be compatible with variable selection and model building in the context of a smoothing spline ANOVA model. Basis pursuit is applied to obtain the optimal decomposition in terms of having the smallest l 1 norm on the coefficients. We use the functional L 1 norm to measure the importance of each component and determine the "threshold" value by a sequential Monte Carlo bootstrap test algorithm. As a generalized LASSOtype method, LBP produces shrinkage estimates for the coefficients, which greatly facilitates the variable selection process, and provides highly interpretable multivariate functional estimates at the same time. To choose the regularization parameters appearing in the LBP models, generalized approximate cross validation (GACV) is derived as a tuning criterion. To make GACV widely applicable to large data sets, its randomized version is proposed as well. A technique "slice modeling" is used to solve the optimization problem and makes the computation more efficient. LBP has great potential for a wide range of research and application areas such as medical studies, and in this paper we apply it to two large ongoing epidemiological studies: the Wisconsin Epidemiological Study of Diabetic Retinopathy (WESDR) and the Beaver Dam Eye Study (BDES).
Bayesian Smoothing and Regression Splines for Measurement Error Problems
 Journal of the American Statistical Association
, 2001
"... In the presence of covariate measurement error, estimating a regression function nonparametrically is extremely dicult, the problem being related to deconvolution. Various frequentist approaches exist for this problem, but to date there has been no Bayesian treatment. In this paper we describe Bayes ..."
Abstract

Cited by 22 (7 self)
 Add to MetaCart
In the presence of covariate measurement error, estimating a regression function nonparametrically is extremely dicult, the problem being related to deconvolution. Various frequentist approaches exist for this problem, but to date there has been no Bayesian treatment. In this paper we describe Bayesian approaches to modeling a exible regression function when the predictor variable is measured with error. The regression function is modeled with smoothing splines and regression P{splines. Two methods are described for exploration of the posterior. The rst is called iterative conditional modes (ICM) and is only partially Bayesian. ICM uses a componentwise maximization routine to nd the mode of the posterior. It also serves to create starting values for the second method, which is fully Bayesian and uses Markov chain Monte Carlo techniques to generate observations from the joint posterior distribution. Using the MCMC approach has the advantage that interval estimates that directly model and adjust for the measurement error are easily calculated. We provide simulations with several nonlinear regression functions and provide an illustrative example. Our simulations indicate that the frequentist mean squared error properties of the fully Bayesian method are better than those of ICM and also of previously proposed frequentist methods, at least in the examples we have studied. KEY WORDS: Bayesian methods; Eciency; Errors in variables; Functional method; Generalized linear models; Kernel regression; Measurement error; Nonparametric regression; P{splines; Regression Splines; SIMEX; Smoothing Splines; Structural modeling. Short title. Nonparametric Regression with Measurement Error Author Aliations Scott M. Berry (Email: scott@berryconsultants.com) is Statistical Scientist,...
Exact Likelihood Ratio Tests for Penalized Splines,” under revision, available at www.orie.cornell.edu/~davidr/papers
 Biometrika
, 2003
"... Penalized splinebased additive models allow a simple mixed model representation where the variance components control departures from linear models. The smoothing parameter is the ratio between the randomcoefficient and error variances and tests for linear regression reduce to tests for zero rando ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
Penalized splinebased additive models allow a simple mixed model representation where the variance components control departures from linear models. The smoothing parameter is the ratio between the randomcoefficient and error variances and tests for linear regression reduce to tests for zero randomcoefficient variances. We propose exact likelihood and restricted likelihood ratio tests, (R)LRTs, for testing polynomial regression versus a general alternative modeled by penalized splines. Their spectral decompositions are used as the basis of fast simulation algorithms. We derive the asymptotic local power properties of (R)LRTs under weak conditions. In particular we characterize the local alternatives that are detected with asymptotic probability 1. Confidence intervals for the smoothing parameter are obtained by inverting the (R)LRT for a fixed smoothing parameter versus a general alternative. We discuss F and R tests and show that ignoring the variability in the smoothing parameter estimator can have a dramatic effect on their null distributions. The power of several known tests is investigated and a small set of tests with good power properties is identified. Some key words: Linear mixed models, penalized splines, smoothing, zero variance components. 2 Ciprian Crainiceanu et al. 1.
Matching Pursuit
, 1993
"... This paper presents a nonparametric penalized likelihood approach for variable selection and model building, called likelihood basis pursuit (LBP). In the setting of a tensor product reproducing kernel Hilbert space, we decompose the log likelihood into the sum of different functional components suc ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
This paper presents a nonparametric penalized likelihood approach for variable selection and model building, called likelihood basis pursuit (LBP). In the setting of a tensor product reproducing kernel Hilbert space, we decompose the log likelihood into the sum of different functional components such as main effects and interactions, with each component represented by appropriate basis functions. Basis functions are chosen to be compatible with variable selection and model building in the context of a smoothing spline ANOVA model. Basis pursuit is applied to obtain the optimal decomposition in terms of having the smallest l1 norm on the coefficients. We use the functional L1 norm to measure the importance of each component and determine the “threshold” value by a sequential Monte Carlo bootstrap test algorithm. As a generalized LASSOtype method, LBP produces shrinkage estimates for the coefficients, which greatly facilitates variable selection process, and provides highly interpretable multivariate functional estimates at the same time. To choose the regularization parameters appearing in the LBP models, generalized approximate cross validation (GACV) is derived as a tuning criterion. To make GACV widely applicable to large data sets, its randomized version is proposed as well. A technique “slice modeling ” is
Confidence Intervals for Nonparametric Curve Estimates Based on Local Smoothing
 J. Am. Stat. Assoc
, 1998
"... Numerous nonparametric regression methods exist which yield consistent estimators of function curves. Often one is also interested in constructing confidence intervals for the unknown function. Pointwise confidence intervals are available using globally crossvalidated smoothing spline (GCV) estim ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
Numerous nonparametric regression methods exist which yield consistent estimators of function curves. Often one is also interested in constructing confidence intervals for the unknown function. Pointwise confidence intervals are available using globally crossvalidated smoothing spline (GCV) estimation. When the function estimate is based on a single global smoothing parameter the resulting confidence intervals may hold their desired confidence level 1 \Gamma ff on average but because bias in nonparametric estimation is not uniform, they do not hold the desired level uniformly at all design points. To deal with this problem, a new smoothing spline estimator is developed which uses a local crossvalidation (LCV) criterion to determine a separate smoothing parameter for each design point. The local smoothing parameters are then used to compute the point estimators of the regression curve and the corresponding pointwise confidence intervals. Incorporation of local information th...
Simple Incorporation of Interactions Into Additive Models
, 2000
"... This article presents penalized spline models that incorporate factor by curve interactions into additive models. A mixed model formulation for penalized splines allows for straightforward model fitting, smoothing parameter selection, and hypothesis testing. We illustrate the proposed model by apply ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
This article presents penalized spline models that incorporate factor by curve interactions into additive models. A mixed model formulation for penalized splines allows for straightforward model fitting, smoothing parameter selection, and hypothesis testing. We illustrate the proposed model by applying it to pollen ragweed data in which seasonal trends vary by year.
Approximating Data with weighted smoothing Splines
, 2009
"... Given a data set (ti,yi), i = 1,...,n with the ti ∈ [0, 1] nonparametric regression is concerned with the problem of specifying a suitable function fn: [0, 1] → R such that the data can be reasonably approximated by the points (ti,fn(ti)), i = 1,...,n. If a data set exhibits large variations in lo ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
Given a data set (ti,yi), i = 1,...,n with the ti ∈ [0, 1] nonparametric regression is concerned with the problem of specifying a suitable function fn: [0, 1] → R such that the data can be reasonably approximated by the points (ti,fn(ti)), i = 1,...,n. If a data set exhibits large variations in local behaviour, for example large peaks as in spectroscopy data, then the method must be able to adapt to the local changes in smoothness. Whilst many methods are able to accomplish this they are less successful at adapting derivatives. In this paper we show how the goal of local adaptivity of the function and its first and second derivatives can be attained in a simple manner using weighted smoothing splines. A residual based concept of approximation is used which forces local adaptivity of the regression function together with a global regularization which makes the function as smooth as possible subject to the approximation constraints.