Results 1  10
of
34
Constructive Incremental Learning from Only Local Information
, 1998
"... ... This article illustrates the potential learning capabilities of purely local learning and offers an interesting and powerful approach to learning with receptive fields. ..."
Abstract

Cited by 160 (37 self)
 Add to MetaCart
... This article illustrates the potential learning capabilities of purely local learning and offers an interesting and powerful approach to learning with receptive fields.
Linear smoothers and additive models
 The Annals of Statistics
, 1989
"... We study linear smoothers and their use in building nonparametric regression models. In part Qfthis paper we examine certain aspects of linear smoothers for scatterplots; examples of these are the running mean and running line, kernel, and cubic spline smoothers. The eigenvalue and singular value d ..."
Abstract

Cited by 70 (2 self)
 Add to MetaCart
We study linear smoothers and their use in building nonparametric regression models. In part Qfthis paper we examine certain aspects of linear smoothers for scatterplots; examples of these are the running mean and running line, kernel, and cubic spline smoothers. The eigenvalue and singular value decompositions of the corresponding smoother matrix are used to qualitatively describe a smoother, and several other topics such as the number of degrees of freedom of a smoother are discussed. In the second part of the paper we describe how Iinearsmoothers can be used to estimate the additive model, a powerful nonparametric regression model, using the "backfitting algorithm". We study the convergence of the backfitting algorithm and prove its convergence for a class of smoothers that includes cubic e:ttJlCl€~nt jJI:::Jll<l.li:6I;:U least squares. algorithm and ' dis.cuss ev'W()r(is: Neaparametric, seanparametric, regression, GaussSeidelalgorithm,
Predictive ApplicationPerformance Modeling in a Computational Grid Environment
, 1999
"... This paper describes and evaluates the application of three local learning algorithms  nearestneighbor, weightedaverage, and locallyweighted polynomial regression  for the prediction of runspecific resourceusage on the basis of runtime input parameters supplied to tools. A twolevel knowl ..."
Abstract

Cited by 60 (12 self)
 Add to MetaCart
This paper describes and evaluates the application of three local learning algorithms  nearestneighbor, weightedaverage, and locallyweighted polynomial regression  for the prediction of runspecific resourceusage on the basis of runtime input parameters supplied to tools. A twolevel knowledge base allows the learning algorithms to track shortterm fluctuations in the performance of computing systems, and the use of instance editing techniques improves the scalability of the performancemodeling system. The learning algorithms assist PUNCH, a networkcomputing system at Purdue University, in emulating an ideal user in terms of its resource management and usage policies. 1. Introduction It is now recognized that the heterogeneous nature of the networkcomputing environment cannot be effectively exploited without some form of adaptive or demanddriven resource management (e.g., [10, 11, 12, 14, 18, 27]). A demanddriven resource management system can be characterized by its a...
The Racing Algorithm: Model Selection for Lazy Learners
 Artificial Intelligence Review
, 1997
"... Given a set of models and some training data, we would like to find the model that best describes the data. Finding the model with the lowest generalization error is a computationally expensive process, especially if the number of testing points is high or if the number of models is large. Optimizat ..."
Abstract

Cited by 50 (3 self)
 Add to MetaCart
Given a set of models and some training data, we would like to find the model that best describes the data. Finding the model with the lowest generalization error is a computationally expensive process, especially if the number of testing points is high or if the number of models is large. Optimization techniques such as hill climbing or genetic algorithms are helpful but can end up with a model that is arbitrarily worse than the best one or cannot be used because there is no distance metric on the space of discrete models. In this paper we develop a technique called "racing" that tests the set of models in parallel, quickly discards those models that are clearly inferior and concentrates the computational effort on differentiating among the better models. Racing is especially suitable for selecting among lazy learners since training requires negligible expense, and incremental testing using leaveoneout cross validation is efficient. We use racing to select among various lazy learnin...
Assessing the quality of learned local models
 Advances in Neural Information Processing Systems 6
, 1994
"... An approach is presented to learning high dimensional functions in the case where the learning algorithm can affect the generation of new data. A local modeling algorithm, locally weighted regression, is used to represent the learned function. Architectural parameters of the approach, such as distan ..."
Abstract

Cited by 43 (15 self)
 Add to MetaCart
An approach is presented to learning high dimensional functions in the case where the learning algorithm can affect the generation of new data. A local modeling algorithm, locally weighted regression, is used to represent the learned function. Architectural parameters of the approach, such as distance metrics, are also localized and become a function of the query point instead of being global. Statistical tests are given for when a local model is good enough and sampling should be moved to a new area. Our methods explicitly deal with the case where prediction accuracy requirements exist during exploration: By gradually shifting a “center of exploration ” and controlling the speed of the shift with local prediction accuracy, a goaldirected exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach with simulation results and results from a real robot learning a complex juggling task. 1
A Monte Carlo study of the forecasting performance of empirical SETAR models
, 1997
"... In this paper we investigate the multiperiod forecast performance of a number of empirical selfexciting threshold autoregressive (SETAR) models that have been proposed in the literature for modelling exchange rates and GNP, amongst other variables. We take each of the empirical SETAR models in turn ..."
Abstract

Cited by 25 (4 self)
 Add to MetaCart
In this paper we investigate the multiperiod forecast performance of a number of empirical selfexciting threshold autoregressive (SETAR) models that have been proposed in the literature for modelling exchange rates and GNP, amongst other variables. We take each of the empirical SETAR models in turn as the DGP to ensure that the `nonlinearity' characterises the future, and compare the forecast performance of SETAR and linear autoregressive models on a number of quantitative and qualitative criteria. Our results indicate that nonlinear models have an edge in certain states of nature but not in others, and that this can be highlighted by evaluating forecasts conditional upon the regime.
Explaining the FavoriteLongshot Bias: Is it RiskLove or Misperceptions?
, 2007
"... The favoritelongshot bias presents a challenge for theories of decision making under uncertainty. This longstanding empirical regularity is that betting odds provide biased estimates of the probability of a horse winning—longshots are overbet, while favorites are underbet. Neoclassical explanations ..."
Abstract

Cited by 22 (5 self)
 Add to MetaCart
The favoritelongshot bias presents a challenge for theories of decision making under uncertainty. This longstanding empirical regularity is that betting odds provide biased estimates of the probability of a horse winning—longshots are overbet, while favorites are underbet. Neoclassical explanations focus on rational gamblers who overbet longshots due to risklove. The competing behavioral explanations emphasize the role of misperceptions of probabilities. We provide novel empirical tests that can discriminate between these competing theories by focusing on the pricing of compound bets. We test whether the models that explain gamblers ’ choices in one part of their choice set (betting to win) can also rationalize decisions over a wider choice set, including compound bets in the exacta, quinella or trifecta pools. Using a new, largescale dataset ideally suited to implement these tests we find evidence in favor of the view that misperceptions of probability drive the favoritelongshot bias, as suggested by Prospect Theory. Along the way we provide more robust evidence on the favoritelongshot bias, falsifying the conventional wisdom that the bias is large enough to yield profit opportunities (it isn’t) and that it becomes more severe in the last race (it doesn’t). ∗We thank David Siegel of Equibase for supplying the data, and Scott Hereld and Ravi Pillai for their
Receptive Field Weighted Regression
, 1997
"... We introduce a constructive, incremental learning system for regression problems that models data by means of spatially localized linear models. In contrast to other approaches, the size and shape of the receptive field of each locally linear model as well as the parameters of the locally linear mod ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
We introduce a constructive, incremental learning system for regression problems that models data by means of spatially localized linear models. In contrast to other approaches, the size and shape of the receptive field of each locally linear model as well as the parameters of the locally linear model itself are learned independently, i.e., without the need for competition or any other kind of communication. This characteristic is accomplished by incrementally minimizing a weighted penalized local cross validation error. As a result, we obtain a learning system that can allocate resources as needed while dealing with the biasvariance dilemma in a principled way. The spatial localization of the linear models increases robustness towards negative interference. Our learning system can be interpreted as a nonparametric adaptive bandwidth smoother, as a mixture of experts where the experts are trained in isolation, and as a learning system which profits from combining independent expert knowledge on the same problem. It illustrates the potential learning capabilities of purely local learning and offers an interesting and powerful approach to learning with receptive fields.
Investment Learning with hierarchical PSOM
, 1995
"... The recently introduced "Parameterized SelfOrganizing Maps" ("PSOM") shows excellent function mapping capabilities after learning of a remarkable small set of training data. This is a very important feature in fields where the acquisition of training data is costly, for example in robotics. As a f ..."
Abstract

Cited by 10 (5 self)
 Add to MetaCart
The recently introduced "Parameterized SelfOrganizing Maps" ("PSOM") shows excellent function mapping capabilities after learning of a remarkable small set of training data. This is a very important feature in fields where the acquisition of training data is costly, for example in robotics. As a first demonstration, we compare results for the task of kinematic mapping of a 3DOF robot finger, obtained by a PSOM and a standard backprop network. A new way of structuring learning becomes feasible: following the idea of interpolating basis mappings learned for a small set of special circumstances, we decompose learning into two phases: (i) In the first investment learning phase we pretrain a hierarchical PSOMnetwork with a set of basis mappings, each capturing a prototypical situation or system context. (ii) Then in the second phase, the mapping "skill" adapts very rapidly, when the system context changes to new, unknown situations. In this paper we demonstrate the potential of this a...
Nonparametric Regression for Learning Nonlinear Transformations
 PRERATIONAL INTELLIGENCE IN STRATEGIES, HIGHLEVEL PROCESSES AND COLLECTIVE BEHAVIOR
"... Information processing in animals and artificial movement systems consists of a series of transformations that map sensory signals to intermediate representations, and finally to motor commands. Given the physical and neuroanatomical differences between individuals and the need for plasticity during ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
Information processing in animals and artificial movement systems consists of a series of transformations that map sensory signals to intermediate representations, and finally to motor commands. Given the physical and neuroanatomical differences between individuals and the need for plasticity during development, it is highly likely that such transformations are learned rather than preprogrammed by evolution. Such selforganizing processes, capable of discovering nonlinear dependencies between different groups of signals, are one essential part of prerational intelligence. While neural network algorithms seem to be the natural choice when searching for solutions for learning transformations, this paper will take a more careful look at which types of neural networks are actually suited for the requirements of an autonomous learning system. The approach that we will pursue is guided by recent developments in learning theory that have linked neural network learning to well established statistical theories. In particular, this new statistical understanding has given rise to the development of neural network systems that are directly based on statistical methods. One family of such methods stems from nonparametric regression. This paper will compare nonparametric learning with the more widely used parametric counterparts in a non technical fashion, and investigate how these two families differ in their properties and their applicabilities. We will argue that nonparametric neural networks offer a set of characteristics that make them a very promising candidate for online learning in autonomous system.