Results 1  10
of
34
Multivariate adaptive regression splines
 The Annals of Statistics
, 1991
"... A new method is presented for flexible regression modeling of high dimensional data. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automaticall ..."
Abstract

Cited by 700 (2 self)
 Add to MetaCart
A new method is presented for flexible regression modeling of high dimensional data. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data. This procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties. Unlike recursive partitioning, however, this method produces continuous models with continuous derivatives. It has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables. In addition, the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions.
Flexible Discriminant Analysis by Optimal Scoring
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
, 1993
"... Fisher's linear discriminant analysis is a valuable tool for multigroup classification. With a large number of predictors, one can nd a reduced number of discriminant coordinate functions that are "optimal" for separating the groups. With two such functions one can produce a classific ..."
Abstract

Cited by 143 (12 self)
 Add to MetaCart
Fisher's linear discriminant analysis is a valuable tool for multigroup classification. With a large number of predictors, one can nd a reduced number of discriminant coordinate functions that are "optimal" for separating the groups. With two such functions one can produce a classification map that partitions the reduced space into regions that are identified with group membership, and the decision boundaries are linear. This paper is about richer nonlinear classification schemes. Linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups. We obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method. In this way, any multiresponse regression technique (such as MARS or neural networks) can be postprocessed to improve their classification performence.
Smoothing Spline ANOVA for Exponential Families, with Application to the Wisconsin Epidemiological Study of Diabetic Retinopathy
 ANN. STATIST
, 1995
"... Let y i ; i = 1; \Delta \Delta \Delta ; n be independent observations with the density of y i of the form h(y i ; f i ) = exp[y i f i \Gammab(f i )+c(y i )], where b and c are given functions and b is twice continuously differentiable and bounded away from 0. Let f i = f(t(i)), where t = (t 1 ; \De ..."
Abstract

Cited by 99 (45 self)
 Add to MetaCart
(Show Context)
Let y i ; i = 1; \Delta \Delta \Delta ; n be independent observations with the density of y i of the form h(y i ; f i ) = exp[y i f i \Gammab(f i )+c(y i )], where b and c are given functions and b is twice continuously differentiable and bounded away from 0. Let f i = f(t(i)), where t = (t 1 ; \Delta \Delta \Delta ; t d ) 2 T (1)\Omega \Delta \Delta \Delta\Omega T (d) = T , the T (ff) are measureable spaces of rather general form, and f is an unknown function on T with some assumed `smoothness' properties. Given fy i ; t(i); i = 1; \Delta \Delta \Delta ; ng, it is desired to estimate f(t) for t in some region of interest contained in T . We develop the fitting of smoothing spline ANOVA models to this data of the form f(t) = C + P ff f ff (t ff ) + P ff!fi f fffi (t ff ; t fi ) + \Delta \Delta \Delta. The components of the decomposition satisfy side conditions which generalize the usual side conditions for parametric ANOVA. The estimate of f is obtained as the minimizer...
Smoothing Spline ANOVA with ComponentWise Bayesian "Confidence Intervals"
 Journal of Computational and Graphical Statistics
, 1992
"... We study a multivariate smoothing spline estimate of a function of several variables, based on an ANOVA decomposition as sums of main effect functions (of one variable), twofactor interaction functions (of two variables), etc. We derive the Bayesian "confidence intervals" for the componen ..."
Abstract

Cited by 52 (21 self)
 Add to MetaCart
(Show Context)
We study a multivariate smoothing spline estimate of a function of several variables, based on an ANOVA decomposition as sums of main effect functions (of one variable), twofactor interaction functions (of two variables), etc. We derive the Bayesian "confidence intervals" for the components of this decomposition and demonstrate that, even with multiple smoothing parameters, they can be efficiently computed using the publicly available code RKPACK, which was originally designed just to compute the estimates. We carry out a small Monte Carlo study to see how closely the actual properties of these componentwise confidence intervals match their nominal confidence levels. Lastly, we analyze some lake acidity data as a function of calcium concentration, latitude, and longitude, using both polynomial and thin plate spline main effects in the same model. KEY WORDS: Bayesian "confidence intervals"; Multivariate function estimation; RKPACK; Smoothing spline ANOVA. Chong Gu chong@pop.stat.pur...
Piecewisepolynomial regression trees
 Statistica Sinica
, 1994
"... A nonparametric function 1 estimation method called SUPPORT (“Smoothed and Unsmoothed PiecewisePolynomial Regression Trees”) is described. The estimate is typically made up of several pieces, each piece being obtained by fitting a polynomial regression to the observations in a subregion of the data ..."
Abstract

Cited by 51 (8 self)
 Add to MetaCart
A nonparametric function 1 estimation method called SUPPORT (“Smoothed and Unsmoothed PiecewisePolynomial Regression Trees”) is described. The estimate is typically made up of several pieces, each piece being obtained by fitting a polynomial regression to the observations in a subregion of the data space. Partitioning is carried out recursively as in a treestructured method. If the estimate is required to be smooth, the polynomial pieces may be glued together by means of weighted averaging. The smoothed estimate is thus obtained in three steps. In the first step, the regressor space is recursively partitioned until the data in each piece are adequately fitted by a polynomial of a fixed order. Partitioning is guided by analysis of the distributions of residuals and crossvalidation estimates of prediction mean square error. In the second step, the data within a neighborhood of each partition are fitted by a polynomial. The final estimate of the regression function is obtained by averaging the polynomial pieces, using smooth weight functions each of which diminishes rapidly to zero outside its associated partition. Estimates of derivatives of the regression function may be
On Growing Better Decision Trees from Data
, 1995
"... This thesis investigates the problem of growing decision trees from data, for the purposes of classification and prediction. ..."
Abstract

Cited by 40 (0 self)
 Add to MetaCart
This thesis investigates the problem of growing decision trees from data, for the purposes of classification and prediction.
Penalized Triograms: Total Variation Regularization for Bivariate Smoothing, preprint
, 2002
"... Abstract. Hansen, Kooperberg, and Sardy (1998) introduced a family of continuous, piecewise linear functions defined over adaptively selected triangulations of the plane as a general approach to statistical modeling of bivariate densities, regression and hazard functions. These triograms enjoy a nat ..."
Abstract

Cited by 28 (4 self)
 Add to MetaCart
Abstract. Hansen, Kooperberg, and Sardy (1998) introduced a family of continuous, piecewise linear functions defined over adaptively selected triangulations of the plane as a general approach to statistical modeling of bivariate densities, regression and hazard functions. These triograms enjoy a natural affine equivariance that offers distinct advantages over competing tensor product methods that are more commonly used in statistical applications. Triograms employ basis functions consisting of linear “tent functions ” defined with respect to a triangulation of a given planar domain. As in knot selection for univariate splines, Hansen, et al adopt the regression spline approach of Stone (1994). Vertices of the triangulation are introduced or removed sequentially in an effort to balance fidelity to the data and parsimony. In this paper we explore a smoothing spline variant of the triogram model based on a roughness penalty adapted to the piecewise linear structure of the triogram model. We show that the proposed roughness penalty may be interpreted as a total variation penalty on the gradient of the fitted function. The methods are illustrated with two artificial examples and with an application to estimated quantile surfaces of land value in the Chicago metropolitan area. “Goniolatry, or the worship of angles,...” Pynchon (1997) 1.
Bivariate Tensorproduct BSplines in a Partly Linear Model
, 1996
"... : In some applications, the mean or median response is linearly related to some variables but the relation to additional variables are not easily parameterized. Partly linear models arise naturally in such circumstances. Suppose that a random sample f(T i ; X i ; Y i ); i = 1; 2; \Delta \Delta \Delt ..."
Abstract

Cited by 24 (4 self)
 Add to MetaCart
: In some applications, the mean or median response is linearly related to some variables but the relation to additional variables are not easily parameterized. Partly linear models arise naturally in such circumstances. Suppose that a random sample f(T i ; X i ; Y i ); i = 1; 2; \Delta \Delta \Delta ; ng is modeled by Y i = X T i fi 0 + g 0 (T i ) + error i , where Y i is a realvalued response, X i 2 R p and T i ranges over a unit square, and g 0 is an unknown function with a certain degree of smoothness. We make use of bivariate tensorproduct Bsplines as an approximation of the function g 0 and consider Mtype regression splines by minimization of P n i=1 ae(Y i \Gamma X T i fi \Gamma g n (T i )) for some convex function ae. Mean, median and quantile regressions are included in this class. We show under appropriate conditions that the parameter estimate of fi achieves its information bound asymptotically and the function estimate of g 0 attains the optimal rate of convergen...
Efficient Algorithms for Function Approximation with Piecewise Linear Sigmoidal Networks
, 1998
"... This paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the wellknown method of fitting the residual. The task of fitting an individual node is accomplished using ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
This paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the wellknown method of fitting the residual. The task of fitting an individual node is accomplished using a new algorithm that searches for the best fit by solving a sequence of Quadratic Programming problems. This approach offers significant advantages over derivativebased search algorithms (e.g. backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stopping criterion, solutions that are independent of initial conditions, good scaling properties and a robust numerical implementation. Empirical results are included to illustrate these characteristics.
Triogram models
 Journal of the American Statistical Association
, 1998
"... In this paper we introduce the Triogram method for function estimation using piecewise linear, bivariate splines based on an adaptively constructed triangulation. We illustrate the technique for bivariate regression and logdensity estimation and indicate how our approach can be applied directly to ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
(Show Context)
In this paper we introduce the Triogram method for function estimation using piecewise linear, bivariate splines based on an adaptively constructed triangulation. We illustrate the technique for bivariate regression and logdensity estimation and indicate how our approach can be applied directly to model bivariate functions in the broader context of an extended linear model. The entire estimation procedure is invariant under a ne transformations and is a natural approach for modeling data when the domain of the predictor variables is a polygonal region in the plane. Although our examples deal exclusively with estimating bivariate functions, the use of Triograms for modeling twofactor interactions in ANOVA decompositions of functions depending on more than two variables is straightforward. Software for tting these models will be available in Version