We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular, standard smoothness functionals lead to a subclass of regularization networks, the well known Radial Basis Functions approximation schemes. This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks. In particular, we introduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same generalization that extends Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman's hinge functions, som...

this paper, we produced the curves fi 0 (E) and

We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular we had discussed how standard smoothness functionals lead to a subclass of regularization networks, the well-known Radial Basis Functions approximation schemes. In this paper weshow that regularization networks encompass amuch broader range of approximation schemes, including many of the popular general additivemodels and some of the neural networks. In particular weintroduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same extension that leads from Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additivemodels to ridge approximation models, containing as special cases Breiman's hinge functions and some forms of Projection Pursuit Regression. We propose to use the term GeneralizedRegularization Networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to differenttypes of smoothness assumptions. In the final part of the paper, weshow the relation between activation functions of the Gaussian and sigmoidal type by considering the simple case of the kernel G(x)=jxj.

We study linear smoothers and their use in building non-parametric regression models. In part Qfthis paper we examine certain aspects of linear smoothers for scatterplots; examples of these are the running mean and running line, kernel, and cubic spline smoothers. The eigenvalue and singular value decompositions of the corresponding smoother matrix are used to qualitatively describe a smoother, and several other topics such as the number of degrees of freedom of a smoother are discussed. In the second part of the paper we describe how Iinear-smoothers can be used to estimate the additive model, a powerful non-parametric regression model, using the "backfitting algorithm". We study the convergence of the backfitting algorithm and prove its convergence for a class of smoothers that includes cubic e:ttJlCl€~nt jJI:::Jll<l.li:6I;:U least squares. algorithm and ' dis.cuss ev'W()r(is: Nea-parametric, sean-parametric, regression, Gauss-Seidelalgorithm,

The typical generalized linear model for a regression of a response Y on predictors (X; Z) has conditional mean function based upon a linear combination of (X; Z). We generalize these models to have a nonparametric component, replacing the linear combination T 0 X + T 0 Z by 0 ( T 0 X) + T 0 Z, where 0 ( ) is an unknown function. We call these generalized partially linear single-index models (GPLSIM). The models include the "single-index" models, which have 0 = 0. Using local linear methods, estimates of the unknown parameters ( 0 ; 0 ) and the unknown function 0 ( ) are proposed, and their asymptotic distributions obtained. Examples illustrate the models and the proposed estimation methodology.

We develop a probability forecasting model through a synthesis of Bayesian beliefnetwork models and classical time-series analysis. By casting Bayesian time-series analyses as temporal belief-network problems, weintroduce dependency models that capture richer and more realistic models of dynamic dependencies. With richer models and associated computational methods, we can movebeyond the rigid classical assumptions of linearityin the relationships among variables and of normality of their probability distributions.

This paper presents a Bayesian approach to binary nonparametric regression which assumes that the argument of the link is an additive function of the explanatory variables and their multiplicative interactions. The paper makes the following contributions. First, a comprehensive approach is presented in which the function estimates are smoothing splines with the smoothing parameters integrated out, and the estimates made robust to outliers. Second, the approach can handle a wide rage of link functions. Third, efficient state space based algorithms are used to carry out the computations. Fourth, an extensive set of simulations is carried out which show that the Bayesian estimator works well and compares favorably to two estimators which are widely used in practice.

We suggest a one dimensional jump detection algorithm based on local polynomial fitting for jumps in regression functions (zero-order jumps) or jumps in derivatives (first-order or higherorder jumps). If jumps exist in the m-th order derivative of the underlying regression function, then an (m + 1) order polynomial is fitted in a neighborhood of each design point. We then characterize the jump information in the coefficients of the highest order terms of the fitted polynomials and suggest an algorithm for jump detection. This method is introduced briefly for the general set-up and then presented in detail for zero-order and first-order jumps. Several simulation examples are discussed. We apply this method to the Bombay (India) sea-level pressure data. Key Words: Nonparametric jump regression model, Jump detection algorithm, Least squares line, Threshold value, Modification procedure, Image processing, Edge detection. 1 Introduction Stock market prices often jump up or down under the in...

this paper we propose and develop a nonparametric multiplicative hazard model that takes into account these aspects. Embedded in the counting process framework, estimation is based on penalized likelihoods and splines. We illustrate our approach by an application to sleep-electroencephalography data with multiple recurrent states of human sleep.

This article describes the analysis of an experiment which was designed to measure the effects of light-dark cycles of varying lengths on sleep patterns in rats. Two inbred strains of rats, albino Lewis and pigmented Brown Norway, were exposed to one of two types of experimental lighting conditions. The first condition, "Baseline," was a standard 24-hour schedule of 12 hours lights on followed by 12 hours lights off; the second condition, "Test," exposed animals to a continuous 3 hour lights on/3 hour lights off schedule. Illuminance level when lights were on varied among three conditions: high (1,000 lux), normal (150 lux) and low (50 lux). Continuous electrophysiological recordings to determine sleep and wake states were performed using standard techniques. Sleep or wake state was determined for each 30 second epoch throughout the experiment. Every 10 consecutive readings were averaged, yielding percentages of time asleep for each day's 288 5-minute intervals. These percentages were in turn averaged over three days of experimentation for each group of rats of the same strain subjected to the same conditions. In this paper, we will concentrate on modeling the percentage of time asleep, which corresponds to total sleep time. The purpose of this analysis is to develop a statistical model for sleep-wake patterns in response to light-dark cycles of varying lengths, which would take into account both the rhythmic circadian component and stimulus-induced shifts in behavior. Such a model could also be used to describe other physiological parameters which may include both rhythmic and acute components, such as temperature and hormone secretion. Studies assessing the effects of acute changes in light conditions often report average data over the period of time in question. Al...