This paper surveys locally weighted learning, a form of lazy learning and memorybased learning, and focuses on locally weighted linear regression. The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning t parameters, interference between old and new data, implementing locally weighted learning e ciently, and applications of locally weighted learning. A companion paper surveys how locally weighted learning can be used in robot learning and control.

Abstract not found

. A kernel smoother is an intuitive estimate of a regression function or conditional expectation; at each point x 0 the estimate of E(Y j x 0 ) is a weighted mean of the sample Y i , with observations close to x 0 receiving the largest weights. Unfortunately this simplicity has flaws. At the boundary of the predictor space, the kernel neighborhood is asymmetric and the estimate may have substantial bias. Bias can be a problem in the interior as well if the predictors are nonuniform or if the regression function has substantial curvature. These problems are particularly severe when the predictors are multidimensional. A variety of kernel modifications have been proposed to provide approximate and asymptotic adjustment for these biases. Such methods generally place substantial restrictions on the regression problems that can be considered; in unfavorable situations, they can perform very poorly. Moreover, the necessary modifications are very difficult to implement in the multidimensional...

this paper we describe two adaptive procedures, one based on C p and the other based on crossvalidation. Still, when we have a final adaptive fit in hand, it is critical to subject it to graphical diagnostics to study its performance. The important implication of these statements is that the above choices must be tailored to each data set in practice; that is, the choices represent a modeling of the data. It is widely accepted that in global parametric regression there are a variety of choices that must be made --- for example, the parametric family to be fitted and the form of the distribution of the response --- and that we must rely on our knowledge of the mechanism generating the data, on model selection diagnostics, and on graphical diagnostic methods to make the choices. The same is true for smoothing. Cleveland (1993) presents many examples of this modeling process. For example, in one application, oxides of nitrogen from an automobile engine are fitted to the equivalence ratio, E, of the fuel and the compression ratio, C, of the engine. Coplots show that it is reasonable to use quadratics as the local parametric family but with the added assumption that given E the fitted f

Abstract not found

this paper is to derive an ecient fully-adaptive procedure for estimating

this paper. We introduce the generalized likelihood statistics to overcome the drawbacks of nonparametric maximum likelihood ratio statistics. New Wilks phenomenon is unveiled. We demonstrate that a class of the generalized likelihood statistics based on some appropriate nonparametric estimators are asymptotically distribution free and follow

were introduced as a means of extending the techniques of ordinary parametric regression to several commonly-used regression models arising from non-normal likelihoods. Typically these models have a variance that depends on the mean function. However, in many cases the likelihood is unknown, but the relationship between mean and variance can be specified. This has led to the consideration of quasi-likelihood methods, where the conditionallog-likelihood is replaced by a quasi-likelihood function. In this article we investigate the extension of the nonparametric regression technique of local polynomial fitting with a kernel weight to these more general contexts. In the ordinary regression case local polynomial fitting has been seen to possess several appealing features in terms of intuitive and mathematical simplicity. One noteworthy feature is the better performance near the boundaries compared to the traditional kernel regression estimators. These properties are shown to carryover to the generalized linear model and quasi-likelihood model. The end result is a class of kernel type estimators for smoothing in quasi-likelihood models. These estimators can be viewed as a straightforward generalization of the usual parametric estimators. In addition, their simple asymptotic distributions allow for simple interpretation

We apply the local linear regression technique for estimation of functional-coefficient regression models for time series data. The models include threshold autoregressive models (Tong 1990) and functional-coefficient autoregressive models (Chen and Tsay 1993) as special cases but with the added advantages such as depicting finer structure of the underlying dynamics and better post-sample forecasting performance. We have also proposed a new bootstrap test for the goodness of fit of models and a bandwidth selector based on newly defined cross-validatory estimation for the expected forecasting errors. The proposed methodology is data-analytic and is of appreciable flexibility to analyze complex and multivariate nonlinear structures without suffering from the "curse of dimensionality". The asymptotic properties of the proposed estimators are investigated under the ff-mixing condition. Both simulated and real data examples are used for illustration. Key Words: ff-mixing; Asymptotic normali...

Local high-order polynomial fitting is employed for the estimation of the multivariate regression function m (x 1 , . . . , x d ) = E [y (Y d ) | X 1 = x 1 , . . . , X d = x d ], and of its partial derivatives, for stationary random processes {Y i , X i }. The function y may be selected to yield estimates of the conditional mean, conditional moments and conditional distributions. Uniform strong consistency over compact subsets of R d , along with rates, are established for the regression function and its partial derivatives for strongly mixing processes. Short Title: Multivariate Regression Estimation. Key Words: Multivariate regression estimation, local polynomial fitting, mixing processes, uniform strong consistency, rates of convergence. AMS (1991) Subject Classification: 62G07, 62H12, 62M09. ################## This work was supported by the Office of Naval Research under Grant N00014-90-J-1175. - 2 - 1. Introduction Let {Y i , X i } i =- be jointly stationary processes on...