Residual Algorithms: Reinforcement Learning with Function Approximation

Residual Algorithms: Reinforcement Learning with Function Approximation (1995)

Cached

Download Links

[www.cs.cmu.edu]
[www.leemon.com]

Other Repositories/Bibliography

DBLP

by Leemon Baird

Venue:	In Proceedings of the Twelfth International Conference on Machine Learning
Citations:	207 - 5 self

Versions

Version History

Metadata Version 3

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	Residual Algorithms: Reinforcement Learning with Function Approximation	user correction - Legacy Corrections
AUTHOR NAME	Leemon Baird	SVM HeaderParse 0.1
AUTHOR AFFIL	Department of Computer Science	SVM HeaderParse 0.2
AUTHOR ADDR	U.S. Air Force Academy, CO 80840-6234	SVM HeaderParse 0.1
ABSTRACT	A number of reinforcement learning algorithms have been developed that are guaranteed to converge to the optimal solution when used with lookup tables. It is shown, however, that these algorithms can easily become unstable when implemented directly with a general function-approximation system, such as a sigmoidal multilayer perceptron, a radial-basisfunction system, a memory-based learning system, or even a linear function-approximation system. A new class of algorithms, residual gradient algorithms, is proposed, which perform gradient descent on the mean squared Bellman residual, guaranteeing convergence. It is shown, however, that they may learn very slowly in some cases. A larger class of algorithms, residual algorithms, is proposed that has the guaranteed convergence of the residual gradient algorithms, yet can retain the fast learning speed of direct algorithms. In fact, both direct and residual gradient algorithms are shown to be special cases of residual algorithms, and it is s...	user correction - Legacy Corrections
YEAR	1995	INFERENCE
VENUE	In Proceedings of the Twelfth International Conference on Machine Learning	INFERENCE
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	30--37	INFERENCE
CITATIONS	11 found	ParsCit 1.0