With ideal spatial adaptation, an oracle furnishes information about how best to adapt a spatially variable estimator, whether piecewise constant, piecewise polynomial, variable knot spline, or variable bandwidth kernel, to the unknown function. Estimation with the aid of an oracle o ers dramatic advantages over traditional linear estimation by nonadaptive kernels � however, it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone. We describe a new principle for spatially-adaptive estimation: selective wavelet reconstruction. Weshowthatvariableknot spline ts and piecewise-polynomial ts, when equipped with an oracle to select the knots, are not dramatically more powerful than selective wavelet reconstruction with an oracle. We develop a practical spatially adaptive method, RiskShrink, which works by shrinkage of empirical wavelet coe cients. RiskShrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so. A new inequality inmultivariate normal decision theory which wecallthe oracle inequality shows that attained performance di ers from ideal performance by at most a factor 2logn, where n is the sample size. Moreover no estimator can give a better guarantee than this. Within the class of spatially adaptive procedures, RiskShrink is essentially optimal. Relying only on the data, it comes within a factor log 2 n of the performance of piecewise polynomial and variable-knot spline methods equipped with an oracle. In contrast, it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone.

This paper surveys locally weighted learning, a form of lazy learning and memorybased learning, and focuses on locally weighted linear regression. The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning t parameters, interference between old and new data, implementing locally weighted learning e ciently, and applications of locally weighted learning. A companion paper surveys how locally weighted learning can be used in robot learning and control.

Abstract not found

. The problem of optimal adaptive estimation of a function at a given point from noisy data is considered. Two procedures are proved to be asymptotically optimal for different settings. First we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel. We propose a bandwidth selection procedure and prove its optimality in the asymptotic sense. Moreover, this optimality is stated not only among kernel estimators with a variable kernel. The resulting estimator is optimal among all feasible estimators. The important feature of this procedure is that no prior information is used about smoothness properties of the estimated function i.e. the procedure is completely adaptive and "works" for the class of all functions. With it the attainable accuracy of estimation depends on the function itself and it is expressed in terms of "ideal" bandwidth corresponding to this function. The second procedure can be considered as a specification of the firs...

Local maximum likelihood estimation is a nonparametric counterpart of the widely-used parametric maximum likelihood technique. It extends the scope of the parametric maximum likelihood method to a much wider class of parametric spaces. Associated with this nonparametric estimation scheme is the issue of bandwidth selection and bias and variance assessment. This article provides a unified approach to selecting a bandwidth and constructing con dence intervals in local maximum likelihood estimation. The approach is then applied to least-squares nonparametric regression and to nonparametric logistic regression. Our experiences in these two settings show that the general idea outlined here is powerful and encouraging.

A decisive question in nonparametric smoothing techniques is the choice of the bandwidth or smoothing parameter. The present paper addresses this question when using local polynomial approximations for estimating the regression function and its derivatives. A fully-automatic bandwidth selection procedure has been proposed by Fan and Gijbels (1995), and the empirical performance of it was tested in detail via a variety of examples. Those experiences supported the methodology towards a great extend. In this paper we establish asymptotic results for the proposed variable bandwidth selector. We provide the rate of convergence of the bandwidth estimate, and obtain the asymptotic distribution of its error relative to the theoretical optimal variable bandwidth. Those asymptotic properties give extra support to the developed bandwidth selection procedure. It is also demonstrated how the proposed selection method can be applied in the density estimation setup. Some examples illustrate this ap...

. We begin by analyzing the local adaptation properties of wavelet-based curve estimators. It is argued that while wavelet methods enjoy outstanding adaptability in terms of the manner in which they capture irregular episodes in a curve, they are not nearly as adaptive when considered from the viewpoint of tracking more subtle changes in a smooth function. We point out that while this problem may be remedied by modifying wavelet estimators, simple modifications are typically not sufficient to properly achieve adaptive smoothing of a relatively highly differentiable function. In that case, local changes to the primary level of resolution of the wavelet transform are required. While such an approach is feasible, it is not an attractive proposition on either practical or aesthetic grounds. Motivated by this difficulty, we develop local versions of familiar smoothing methods, such as cross-validation and smoothed cross-validation, in the contexts of density estimation and regression. It is...

The prices of index options at a given date are usually represented via the corresponding implied volatility surface, presenting skew/smile features and term structure which several models have attempted to reproduce.

We study a robust version of local linear regression smoothers augmented with variable bandwidth. The proposed method inherits the advantages of local polynomial regression and overcomes lack of robustness of least-squares techniques. The use of variable bandwidth enhances the flexibility of the resulting local M-estimators and makes them possible to cope well with spatially inhomogeneous curves, heteroscedastic errors and nonuniform design densities. Under appropriate regularity conditions, it is shown that the proposed estimators exist and are asymptotically normal. Based on the robust estimation equation, we introduce one-step local M-estimators to reduce computational burden. It is demonstrated that the one-step local Mestimators share the same asymptotic distributions as the fully iterative M-estimators, as long as the initial estimators are good enough. In other words, the one-step local M-estimators reduce significantly the computation cost of the fully iterative M-estimators wi...

A hybrid estimator of the log-spectral density of a stationary time series is proposed. First, a multiple taper estimate is performed, followed by kernel smoothing the log-multiple taper estimate. This procedure reduces the expected mean square error by ( ß 2 4 ) 4=5 over simply smoothing the log tapered periodogram. A data adaptive implementation of a variable bandwidth kernel smoother is given. 1 INTRODUCTION We consider a discrete, stationary, Gaussian time series fx j ; j = 1; : : : Ng with a smooth spectral density, S(f ), which is bounded away from zero. The autocovariance is the Fourier transform of the spectral density: Cov [x j ; x k ] = R 1 2 \Gamma 1 2 S(f)e 2ßi(j \Gammak)f df . When the logarithm of the spectral density, `(f) j ln[S(f )], is desired, two common approaches are: 1) to estimate the spectral density and then transform to the logarithm; and 2) to smooth the logarithm of the tapered periodogram. The first approach can be sensitive to broad-band bias ...