Exploration and apprenticeship learning in reinforcement learning (2005)
Cached
Download Links
- [imls.engr.oregonstate.edu]
- [www.robotics.stanford.edu]
- [ai.stanford.edu]
- [www.robotics.stanford.edu]
- CiteULike
- DBLP
Other Repositories/Bibliography
| Venue: | in Proc. 21st International Conference on Machine Learning |
| Citations: | 52 - 2 self |
BibTeX
@INPROCEEDINGS{Abbeel05explorationand,
author = {Pieter Abbeel and Andrew Y. Ng},
title = {Exploration and apprenticeship learning in reinforcement learning},
booktitle = {in Proc. 21st International Conference on Machine Learning},
year = {2005},
pages = {1--8},
publisher = {ICML}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
We consider reinforcement learning in systems with unknown dynamics. Algorithms such as E 3 (Kearns and Singh, 2002) learn near-optimal policies by using “exploration policies ” to drive the system towards poorly modeled states, so as to encourage exploration. But this makes these algorithms impractical for many systems; for example, on an autonomous helicopter, overly aggressive exploration may well result in a crash. In this paper, we consider the apprenticeship learning setting in which a teacher demonstration of the task is available. We show that, given the initial demonstration, no explicit exploration is necessary, and we can attain near-optimal performance (compared to the teacher) simply by repeatedly executing “exploitation policies ” that try to maximize rewards. In finite-state MDPs, our algorithm scales polynomially in the number of states; in continuous-state linear dynamical systems, it scales polynomially in the dimension of the state. These results are proved using a martingale construction over relative losses. 1.
