Learning Without State-Estimation in Partially Observable Markovian Decision Processes (1994)
Cached
Download Links
- [ftp.cs.colorado.edu]
- [www.eecs.umich.edu]
- DBLP
Other Repositories/Bibliography
| Venue: | In Proceedings of the Eleventh International Conference on Machine Learning |
| Citations: | 115 - 5 self |
BibTeX
@INPROCEEDINGS{Singh94learningwithout,
author = {Satinder P. Singh and Tommi Jaakkola and Michael I. Jordan},
title = {Learning Without State-Estimation in Partially Observable Markovian Decision Processes},
booktitle = {In Proceedings of the Eleventh International Conference on Machine Learning},
year = {1994},
pages = {284--292},
publisher = {Morgan Kaufmann}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Reinforcement learning (RL) algorithms provide a sound theoretical basis for building learning control architectures for embedded agents. Unfortunately all of the theory and much of the practice (see Barto et al., 1983, for an exception) of RL is limited to Markovian decision processes (MDPs). Many realworld decision tasks, however, are inherently non-Markovian, i.e., the state of the environment is only incompletely known to the learning agent. In this paper we consider only partially observable MDPs (POMDPs), a useful class of non-Markovian decision processes. Most previous approaches to such problems have combined computationally expensive state-estimation techniques with learning control. This paper investigates learning in POMDPs without resorting to any form of state estimation. We present results about what TD(0) and Q-learning will do when applied to POMDPs. It is shown that the conventional discounted RL framework is inadequate to deal with POMDPs. Finally we develop a new fr...
