Recent proposals for implementation of kernel based nonparametric curve estimators are seen to be faster than naive direct implementations by factors up into the hundreds. The main ideas behind two different approaches of this type are made clear. Careful speed comparisons in a variety of settings, and using a variety of machines and software is done. Various issues on computational accuracy and stability are also discussed. The fast methods are seen to be somewhat better than methods traditionally considered very fast, such as LOWESS and smoothing splines. 1

We introduce a class of models for an additive decomposition of groups of curves strati ed by crossed and nested factors, generalizing smoothing splines to such samples by associating them with a corresponding mixed e ects model. The models are also useful for imputation of missing data and exploratory analysis of variance. We prove that the best linear unbiased predictors (BLUP) from the extended mixed e ects model correspond to solutions of a generalized penalized regression where smoothing parameters are directly related to variance components, and we show that these solutions are natural cubic splines. The model parameters are estimated using a highly e cient implementation of the EM algorithm for restricted maximum likelihood (REML) estimation based on a preliminary eigenvector decomposition. Variability of computed estimates can be assessed with asymptotic techniques or with a novel hierarchical bootstrap resampling scheme for nested mixed e ects models. Our methods are applied to menstrual cycle data from studies of reproductive function that measure daily urinary progesterone; the sample of progesterone curves is strati ed by cycles nested within subjects nested within conceptive and non-conceptive groups.

Variation is a key concern in semiconductor manufacturing and is manifest in several forms. Spatial variation across each wafer results from equipment or process limitations, and variation within each die may be exacerbated further by complex pattern dependencies. Spatial variation information is important not only for process optimization and control, but also for design of circuits that are robust to such variation. Systematic and random components of the variation must be identified, and models relating the spatial variation to specific process and pattern causes are needed. In this work, extraction and modeling methods are described for wafer-level, die-level, and wafer--die interaction contributions to spatial variation. Waferlevel estimation methods include filtering, spline, and regression based approaches. Die-level (or intra-die) variation can be extracted using spatial Fourier transform methods; important issues include spectral interpolation and sampling requirements. Finally, the interaction between wafer- and die-level effects is important to fully capture and separate systematic versus random variation; spline- and frequency-based methods are proposed for this modeling. Together, these provide an effective collection of methods to identify and model spatial variation for future use in process control to reduce systematic variation, and in process/device design to produce more robust circuits.

were introduced as a means of extending the techniques of ordinary parametric regression to several commonly-used regression models arising from non-normal likelihoods. Typically these models have a variance that depends on the mean function. However, in many cases the likelihood is unknown, but the relationship between mean and variance can be specified. This has led to the consideration of quasi-likelihood methods, where the conditionallog-likelihood is replaced by a quasi-likelihood function. In this article we investigate the extension of the nonparametric regression technique of local polynomial fitting with a kernel weight to these more general contexts. In the ordinary regression case local polynomial fitting has been seen to possess several appealing features in terms of intuitive and mathematical simplicity. One noteworthy feature is the better performance near the boundaries compared to the traditional kernel regression estimators. These properties are shown to carryover to the generalized linear model and quasi-likelihood model. The end result is a class of kernel type estimators for smoothing in quasi-likelihood models. These estimators can be viewed as a straightforward generalization of the usual parametric estimators. In addition, their simple asymptotic distributions allow for simple interpretation

This paper discusses the intimate relationships between the supervised learning frameworks mentioned in the title. In particular, it shows how all those frameworks can be viewed as particular instances of a single overarching formalism. In doing this many commonly misunderstood aspects of those frameworks are explored. In addition the strengths and weaknesses of those frameworks are compared, and some novel frameworks are suggested (resulting, for example, in a "correction" to the familiar bias-plus-variance formula).

. The theory of wavelets is a developing branch of mathematics with a wide range of potential applications. Compactly supported wavelets are particularly interesting because of their natural ability to represent data with intrinsically local properties. They are useful for the detection of edges and singularities in image and sound analysis, and for data compression. However, most of the wavelet based procedures currently available do not explicitly account for the presence of noise in the data. A discussion of how this can be done in the setting of some simple nonparametric curve estimation problems is given. Wavelet analogues of some familiar kernel and orthogonal series estimators are introduced and their finite sample and asymptotic properties are studied. We discover that there is a fundamental instability in the asymptotic variance of wavelet estimators caused by the lack of translation invariance of the wavelet transform. This is related to the properties of certain lacunary seq...

this paper (available on the World-Wide Web; see our home pages) contains pseudo-code of an efficient implementation. It is based on fast multiplication of the Hessian and a vector due to Pearlmutter (1994) and Mller (1993). Acknowledgments

In this paper, a method for estimating task execution times is presented, in order to facilitate dynamic scheduling in a heterogeneous metacomputing environment. Execution time is treated as a random variable and is statistically estimated from past observations. This method predicts the execution time as a function of several parameters of the input data, and does not require any direct information about the algorithms used by the tasks or the architecture of the machines. Techniques based upon the concept of analytic benchmarking/code profiling [7] are used to accurately determine the performance differences between machines, allowing observations to be shared between machines. Experimental results using real data are presented.

Penalized splines, or P-splines, are regression splines fit by least-squares with a roughness penaly. P-splines have much in common with smoothing splines, but the type of penalty used with a P-spline is somewhat more general than for a smoothing spline. Also, the number and location of the knots of a P-spline is not fixed as with a smoothing spline. Generally, the knots of a P-spline are at fixed quantiles of the independent variable and the only tuning parameter to choose is the number of knots. In this article, the effects of the number of knots on the performance of P-splines are studied. Two algorithms are proposed for the automatic selection of the number of knots. The myoptic algorithm stops when no improvement in the generalized cross validation statistic (GCV) is noticed with the last increase in the number of knots. The full search examines all candidates in a fixed sequence of possible numbers of knots and chooses the candidate that minimizes GCV. The myoptic algo...

We study spline fitting with a roughness penalty that adapts to spatial heterogene-ity in the regression function. Our estimates are pth degree piecewise polynomials with p − 1 continuous derivatives. A large and fixed number of knots is used and smoothing is achieved by putting a quadratic penalty on the jumps of the pth derivative at the knots. To be spatially adaptive, the logarithm of the penalty is itself a linear spline but with relatively few knots and with values at the knots chosen to minimize GCV. This locally-adaptive spline estimator is compared with other spline estimators in the liter-ature such as cubic smoothing splines and knot-selection techniques for least-squares regression. Our estimator can be interpreted as an empirical Bayes estimate for a prior allowing spatial heterogeneity. In cases of spatially heterogeneous regression functions,