Results 1 
5 of
5
How Dependencies between Successive Examples Affect OnLine Learning
, 1996
"... . We study the dynamics of online learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. W ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
. We study the dynamics of online learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. The representation error measures how well the environment is represented by the network after learning. The prediction error is the average error which a continually learning network makes on the next example. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation, the higher the representation error, i.e. the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau. Plateaus are flat spots on the error surface, which can severely slow down the learning proce...
Online Learning with TimeCorrelated Examples
, 1998
"... We study the dynamics of online learning with timecorrelated patterns. In this, we make a distinction between "small" networks and "large" networks. "Small" networks have a finite number of input units and are usually studied using tools from stochastic approximation theory in the limit of small l ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We study the dynamics of online learning with timecorrelated patterns. In this, we make a distinction between "small" networks and "large" networks. "Small" networks have a finite number of input units and are usually studied using tools from stochastic approximation theory in the limit of small learning parameters. "Large" networks have an extensive number of input units. A description in terms of individual weights is no longer useful and tools from statistical mechanics can be applied to compute the evolution of macroscopic order parameters. We give general derivations for both cases, but in the end focus on the effect of correlations on plateaus. Plateaus are long time spans in which the performance of the networks hardly changes. Learning in both "small" and "large" multilayered perceptrons is often hampered by the presence of plateaus. The effect of correlations, however, appears to be quite different: they can have a huge beneficial effect in small networks, but seem to have ...
Presentation Order and onLine Learning
"... We study the effect of the presentation order of training patterns on the performance of online learning neural networks. In the context of time series, we discuss the difference between randomized and natural learning. With regard to learning in cycles, we quantify and compare the performance of a ..."
Abstract
 Add to MetaCart
We study the effect of the presentation order of training patterns on the performance of online learning neural networks. In the context of time series, we discuss the difference between randomized and natural learning. With regard to learning in cycles, we quantify and compare the performance of almost cyclic and purely cyclic learning. 1 Introduction Learning plays a crucial role in most neuralnetwork applications. Through learning the network weights are adapted to meet the requirements of the environment. Usually, the designer has access to a finite number of examples from this environment: the training set. A popular learning strategy is online learning: at each learning step one of the patterns is drawn from the training set and presented to the network leading to a learning step of the form \Deltaw(n) = w(n + 1) \Gamma w(n) = jf(w(n); x(n)) ; (1) with w(n) the weight vector at iteration step n, j the learning parameter, x(n) the presented training pattern, and f(\Delta; \Del...
Online Learning From Finite Training Sets and
 Neural Computation
, 1998
"... this paper, we give an exact analysis of online learning in a simple model system. Our aim is twofold: (1) to assess how the combination of noninfinitesimal learning rates j and finite training sets (containing ff examples per weight) affects online learning, and (2) to compare the generalization p ..."
Abstract
 Add to MetaCart
this paper, we give an exact analysis of online learning in a simple model system. Our aim is twofold: (1) to assess how the combination of noninfinitesimal learning rates j and finite training sets (containing ff examples per weight) affects online learning, and (2) to compare the generalization performance of online and offline learning. A priori, one Online learning can also be used to learn teacher rules that vary in time. The assumption of an infinite set (or `stream') of training examples is then much more plausible, and in fact necessary for continued adaptation of the student. We do not consider this case in the following
Feature
"... selection with equalized salience measures and its application to segmentation ..."
Abstract
 Add to MetaCart
selection with equalized salience measures and its application to segmentation