Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (1999)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Richard Sutton , Doina Precup , Satinder Singh

Venue:	Artificial Intelligence
Citations:	342 - 22 self

Versions

Version History

Metadata Version 3

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning	user correction - Legacy Corrections
AUTHOR NAME	Richard Sutton	user correction - Legacy Corrections
AUTHOR AFFIL	University of Massachusetts, Amherst	user correction - Legacy Corrections
AUTHOR NAME	Doina Precup	user correction - Legacy Corrections
AUTHOR AFFIL	University of Massachusetts, Amherst	user correction - Legacy Corrections
AUTHOR NAME	Satinder Singh	user correction - Legacy Corrections
AUTHOR AFFIL	University of Colorado, Boulder	user correction - Legacy Corrections
ABSTRACT	Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options---closed-loop policies for taking action over a period of time. Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint torques. Overall, we show that options enable temporally abstract knowledge and action to be included in the reinforcement learning framework in a natural and general way. In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning.	user correction - Legacy Corrections
YEAR	1999	INFERENCE
VENUE	Artificial Intelligence	INFERENCE
VENUE TYPE	JOURNAL	INFERENCE
PAGES	181--211	INFERENCE
VOLUME	112	INFERENCE
CITATIONS	76 found	ParsCit 1.0