Results 1  10
of
62
Priors, Stabilizers and Basis Functions: from regularization to radial, tensor and additive splines
, 1993
"... We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular we had discussed how standard smoothness functionals lead to a subclass of regularization networks, th ..."
Abstract

Cited by 79 (14 self)
 Add to MetaCart
We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular we had discussed how standard smoothness functionals lead to a subclass of regularization networks, the wellknown Radial Basis Functions approximation schemes. In this paper weshow that regularization networks encompass amuch broader range of approximation schemes, including many of the popular general additivemodels and some of the neural networks. In particular weintroduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same extension that leads from Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additivemodels to ridge approximation models, containing as special cases Breiman's hinge functions and some forms of Projection Pursuit Regression. We propose to use the term GeneralizedRegularization Networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to differenttypes of smoothness assumptions. In the final part of the paper, weshow the relation between activation functions of the Gaussian and sigmoidal type by considering the simple case of the kernel G(x)=x. In summary,
Fitting a Bivariate Additive Model by Local Polynomial Regression
, 1996
"... While the additive model is a popular nonparametric regression method, many of its theoretical properties are not well understood, especially when the backfitting algorithm is used for computation of the the estimators. This article explores those properties when the additive model is fitted by loca ..."
Abstract

Cited by 44 (11 self)
 Add to MetaCart
While the additive model is a popular nonparametric regression method, many of its theoretical properties are not well understood, especially when the backfitting algorithm is used for computation of the the estimators. This article explores those properties when the additive model is fitted by local polynomial regression. Sufficient conditions guaranteeing the asymptotic existence of unique estimators for the bivariate additive model are given. Asymptotic approximations to the bias and the variance of a homoskedastic bivariate additive model with local polynomial terms are computed. This model is shown to have the same rate of convergence as that of univariate local polynomial regression. We also investigate the estimation of derivatives of the additive component functions.
Smoothing Spline ANOVA with ComponentWise Bayesian "Confidence Intervals"
 Journal of Computational and Graphical Statistics
, 1992
"... We study a multivariate smoothing spline estimate of a function of several variables, based on an ANOVA decomposition as sums of main effect functions (of one variable), twofactor interaction functions (of two variables), etc. We derive the Bayesian "confidence intervals" for the components of this ..."
Abstract

Cited by 44 (17 self)
 Add to MetaCart
We study a multivariate smoothing spline estimate of a function of several variables, based on an ANOVA decomposition as sums of main effect functions (of one variable), twofactor interaction functions (of two variables), etc. We derive the Bayesian "confidence intervals" for the components of this decomposition and demonstrate that, even with multiple smoothing parameters, they can be efficiently computed using the publicly available code RKPACK, which was originally designed just to compute the estimates. We carry out a small Monte Carlo study to see how closely the actual properties of these componentwise confidence intervals match their nominal confidence levels. Lastly, we analyze some lake acidity data as a function of calcium concentration, latitude, and longitude, using both polynomial and thin plate spline main effects in the same model. KEY WORDS: Bayesian "confidence intervals"; Multivariate function estimation; RKPACK; Smoothing spline ANOVA. Chong Gu chong@pop.stat.pur...
G: Minimizing GCV/GML scores with multiple smoothing parameters via the Newton method
 Siam J Sci Statist Comp
, 1991
"... Abstract. The (modified) Newton method is adapted to optimize generalized cross validation (GCV) and generalized maximum likelihood (GML) scores with multiple smoothing parameters. The main concerns in solving the optimization problem are the speed and the reliability of the algorithm, as well as th ..."
Abstract

Cited by 43 (8 self)
 Add to MetaCart
Abstract. The (modified) Newton method is adapted to optimize generalized cross validation (GCV) and generalized maximum likelihood (GML) scores with multiple smoothing parameters. The main concerns in solving the optimization problem are the speed and the reliability of the algorithm, as well as the invariance of the algorithm under transformations under which the problem itself is invariant. The proposed algorithm is believed to be highly efficient for the problem, though it is still rather expensive for large data sets, since its operational counts are (2/3)kn + O(n2), with k the number of smoothing parameters and n the number of observations. Sensible procedures for computing good starting values are also proposed, which should help in keeping the execution load to the minimum possible. The algorithm is implemented in Rkpack [RKPACK and its applications: Fitting smoothing spline models, Tech. Report 857, Department of Statistics, University of Wisconsin, Madison, WI, 1989] and illustrated by examples of fitting additive and interaction spline models. It is noted that the algorithm can also be applied to the maximum likelihood (ML) and the restricted maximum likelihood (REML) estimation of the variance component models.
Nonparametric Estimation and Testing of Interaction in Additive Models
, 2002
"... We consider an additive model with second order interaction terms. Both marginal integration estimators and a combined backfittingintegration estimator are proposed for all components of the model and their derivatives. The corresponding asymptotic distributions are derived. Moreover, two test stat ..."
Abstract

Cited by 21 (7 self)
 Add to MetaCart
We consider an additive model with second order interaction terms. Both marginal integration estimators and a combined backfittingintegration estimator are proposed for all components of the model and their derivatives. The corresponding asymptotic distributions are derived. Moreover, two test statistics for testing the presence of interactions are proposed. Asymptotics for the test functions and local power results are obtained. Since direct implementation of the test procedure based on the asymptotics would produce inaccurate results unless the number of observations is very large, a bootstrap procedure is provided, which is applicable for small or moderate sample sizes. Further, based on these methods a general test for additivity is developed. Estimation and testing methods are shown to work well in simulation studies. Finally, our methods
Asymptotic Properties of Backfitting Estimators
 Journal of Multivariate Analysis
, 1998
"... When additive models with more than two covariates are fitted with the backfitting algorithm proposed by Buja et al. [2], the lack of explicit expressions for the estimators makes study of their theoretical properties cumbersome. Recursion provides a convenient way to extend existing theoretical res ..."
Abstract

Cited by 20 (3 self)
 Add to MetaCart
When additive models with more than two covariates are fitted with the backfitting algorithm proposed by Buja et al. [2], the lack of explicit expressions for the estimators makes study of their theoretical properties cumbersome. Recursion provides a convenient way to extend existing theoretical results for bivariate additive models to models of arbitrary dimension. In the case of local polynomial regression smoothers, recursive asymptotic bias and variance expressions for the backfitting estimators are derived. The estimators are shown to achieve the same rate of convergence as those of univariate local polynomial regression. In the case of independence between the covariates, nonrecursive bias and variance expressions, as well as the asymptotically optimal values for the bandwidth parameters, are provided. 1 Introduction The additive model, originally suggested by Friedman and Stuetzle [4], assumes that the conditional expectation function of the dependent variable can be written a...
A Fully Automated Bandwidth Selection Method for Fitting Additive Models
, 1996
"... This article describes a fully automated bandwidth selection method for additive models that is applicable to the widely used backfitting algorithm of Buja, Hastie and Tibshirani (1989) and does not rely on crossvalidation. The proposed plugin estimator is an extension of the local linear regressi ..."
Abstract

Cited by 18 (7 self)
 Add to MetaCart
This article describes a fully automated bandwidth selection method for additive models that is applicable to the widely used backfitting algorithm of Buja, Hastie and Tibshirani (1989) and does not rely on crossvalidation. The proposed plugin estimator is an extension of the local linear regression estimator of Ruppert, Sheather and Wand (1996) and is shown to achieve the same O p (n \Gamma2=7 ) relative convergence rate for bivariate additive models. If more than two covariates are present, theoretical justification of the method requires independence of the covariates, but simulation experiments show that in practice the method is very robust to violations of this assumption. The behavior of the method is demonstrated on a real dataset. 1 Introduction Additive models (Hastie and Tibshirani [16]) are a popular multivariate nonparametric fitting technique. The additive model assumes that the conditional expectation function of the dependent variable Y can be written as the sum ...
Smoothing Spline ANOVA Fits for Very Large, Nearly Regular Data Sets, with Application to Historical Global Climate Data
, 1995
"... ... validation (GCV), provided that matrix decompositions of size n \Theta n can be carried out, where n is the sample size. We review the randomized trace technique and the backfitting algorithm, and remark that they can be combined to solve the variational problem while choosing the smoothing para ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
... validation (GCV), provided that matrix decompositions of size n \Theta n can be carried out, where n is the sample size. We review the randomized trace technique and the backfitting algorithm, and remark that they can be combined to solve the variational problem while choosing the smoothing parameters by GCV for data sets that are much too large to use matrix decomposition methods directly. Some intermediate calculations to speed up the backfitting algorithm are given which are useful when the data has a tensor product structure. We describe an imputation procedure which can take advantage of data with a (nearly) tensor product structure. As an illustration of an application we discuss the algorithm in the context of fitting and smoothing historical global winter mean surface temperature data and examining the main effects and interactions for time and space.
Estimating Lyapunov Exponents with Nonparametric Regression
, 1990
"... We discuss procedures based on nonparametric regression for estimating the dominant Lyapunov exponent Al from timeseries data generated by a system x t ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
We discuss procedures based on nonparametric regression for estimating the dominant Lyapunov exponent Al from timeseries data generated by a system x t
An iterative algorithm for extending learners to a semisupervised setting
 The 2007 Joint Statistical Meetings (JSM
, 2007
"... In this paper, we present an iterative selftraining algorithm, whose objective is to extend learners from a supervised setting into a semisupervised setting. The algorithm is based on using the predicted values for observations where the response is missing (unlabeled data) and then incorporates t ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
In this paper, we present an iterative selftraining algorithm, whose objective is to extend learners from a supervised setting into a semisupervised setting. The algorithm is based on using the predicted values for observations where the response is missing (unlabeled data) and then incorporates the predictions appropriately at subsequent stages. Convergence properties of the algorithm are investigated for particular learners, such as linear/logistic regression and linear smoothers with particular emphasis on kernel smoothers. Further, implementation issues of the algorithm with other learners such as generalized additive models, tree partitioning methods, partial least squares, etc. are also addressed. The connection between the proposed algorithm and graphbased semisupervised learning methods is also discussed. The algorithm is illustrated on a number of real data sets using a varying degree of labeled responses. Keywords: Semisupervised learning, linear smoothers, convergence, iterative algorithm