DMCA
Hybrid Learning Algorithms for Neural Networks - The adaptive Inclusion of Second Order Information (1996)
| Citations: | 2 - 0 self |
BibTeX
@TECHREPORT{Pfister96hybridlearning,
author = {Marcus Pfister and Raúl Rojas},
title = {Hybrid Learning Algorithms for Neural Networks - The adaptive Inclusion of Second Order Information},
institution = {},
year = {1996}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Since the introduction of the backpropagation algorithm as a learning rule for neural networks much effort has been spent trying to develop faster alternatives. This is e.g. done by using adaptively changing learning rates or exploiting second order information of the error surface. These optimization strategies are fixed once chosen, so if the heuristic does not fit the actual shape of the error surface, the computed weight changes will be far from the optimal ones. In this paper we propose two hybrid learning algorithms, which dynamically switch between different optimization strategies. The algorithms basically use adaptive step sizes for the weight changes, but adaptively include second order information if a valley of the error function is reached. The proposed hybrid algorithms, as well as standard backpropagation and three other known fast learning algorithms, were implemented on a SIMD neurocomputer, Adaptive Solutions CNAPS, and benchmarked against the Carnegie-Mellon benchmar...
Keyphrases
second order information neural network adaptive inclusion hybrid learning algorithm error surface adaptive solution cnaps adaptive step size standard backpropagation optimization strategy error function backpropagation algorithm actual shape hybrid algorithm learning rule optimal one neural network much effort computed weight change simd neurocomputer learning algorithm carnegie-mellon benchmar weight change different optimization strategy
