Abstract

I present computational results suggesting that gainadaptation algorithms based in part on connectionist learning methods may improve over least squares and other classical parameter-estimation methods for stochastic time-varying linear systems. The new algorithms are evaluated with respect to classical methods along three dimensions: asymptotic error, computational complexity, and required prior knowledge about the system. The new algorithms are all of the same order of complexity as LMS methods, O(n), where n is the dimensionality of the system, whereas least-squares methods and the Kalman filter are O(n 2 ). The new methods also improve over the Kalman filter in that they do not require a complete statistical model of how the system varies over time. In a simple computational experiment, the new methods are shown to produce asymptotic error levels near that of the optimal Kalman filter and significantly below those of least-squares and LMS methods. The new methods may perform bet...

Citations

283	Increased rates of convergence through learning rate adaptation - Jacobs
178	Adaptive Filtering Prediction and Control - Goodwin, Sin - 1985
62	Adapting bias by gradient descent: An incremental version of deltabar-delta - Sutton - 1992
27	Practical characteristics of neural network and conventional pattern classifiers on artificial an speech problems - Lippmann, Lee - 1990
20	Accelerated stochastic approximation - Kesten - 1958
13	Learning polynomial functions by feature construction - Sutton - 1991
9	Iterative construction of sparse polynomial approximations - Sanger, Sutton, et al. - 1992
5	Basis-function trees as a generalization of local variable selection methods for function approximation - Sanger - 1991
2	Adaptation of cue-specific learning rates in adaptive networks: Computational and psychological perspectives - Gluck, Glauthier - 1992