Abstract

Since the presentation of the backpropagation algorithm [1] a vast variety of improvements of the technique for training the weights in a feed-forward neural network have been proposed. The following article introduces the concept of supervised learning in multi-layer perceptrons based on the technique of gradient descent. Some problems and drawbacks of the original backpropagation learning procedure are discussed, eventually leading to the development of more sophisticated techniques. This article concentrates on adaptive learning strategies. Some of the most popular learning algorithms are described and discussed according to their classification in terms of global and local adaptation strategies. The behavior of several learning procedures on some popular benchmark problems is reported, thereby illuminating convergence, robustness, and scaling properties of the respective algorithms. I. Introduction At present, supervised learning is probably the most frequently used technique in ...

Citations

2304	Learning Internal Representations by Error Propagation - Rumelhart, Hinton, et al. - 1986
2	Improved convergence rate of backpropagation with dynamic adaptation of the learning rate - Salomon