Learning to predict by the methods of temporal differences

Learning to predict by the methods of temporal differences (1988)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Richard S. Sutton

Venue:	MACHINE LEARNING
Citations:	1060 - 33 self

Versions

Version History

Metadata Version 4

User correction supplied by jfernandez

Datum	Value	Source
TITLE	Learning to predict by the methods of temporal differences	user correction
AUTHOR NAME	Richard S. Sutton	user correction
AUTHOR AFFIL	GTE Laboratories Incorporated	user correction
AUTHOR ADDR	40 Sylvan Road, Waltham, MA 02254, U.S.A.	user correction
ABSTRACT	This article introduces a class of incremental learning procedures specialized for prediction – that is, for using past experience with an incompletely known system to predict its future behavior. Whereas conventional prediction-learning methods assign credit by means of the difference between predicted and actual outcomes, the new methods assign credit by means of the difference between temporally successive predictions. Although such temporal-difference methods have been used in Samuel's checker player, Holland's bucket brigade, and the author's Adaptive Heuristic Critic, they have remained poorly understood. Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods. For most real-world prediction problems, temporal-difference methods require less memory and less peak computation than conventional methods and they produce more accurate predictions. We argue that most problems to which supervised learning is currently applied are really prediction problems of the sort to which temporal-difference methods can be applied to advantage.	user correction
YEAR	1988	INFERENCE
VENUE	MACHINE LEARNING	user correction
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	9--44	INFERENCE
VOLUME	3	INFERENCE
NUMBER	1	user correction
CITATIONS	28 found	ParsCit 1.0