DMCA
PEGASUS: A policy search method for large MDPs and POMDPs (2000)
Cached
Download Links
| Venue: | In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence |
| Citations: | 255 - 9 self |
BibTeX
@INPROCEEDINGS{Ng00pegasus:a,
author = {Andrew Ng and Michael Jordan},
title = {PEGASUS: A policy search method for large MDPs and POMDPs},
booktitle = {In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence},
year = {2000},
pages = {406--415}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We propose a new approach to the problem of searching a space of policies for a Markov decision process (MDP) or a partially observable Markov decision process (POMDP), given a model. Our approach is based on the following observation: Any (PO)MDP can be transformed into an "equivalent" POMDP in which all state transitions (given the current state and action) are deterministic. This reduces the general problem of policy search to one in which we need only consider POMDPs with deterministic transitions. We give a natural way of estimating the value of all policies in these transformed POMDPs. Policy search is then simply performed by searching for a policy with high estimated value. We also establish conditions under which our value estimates will be good, recovering theoretical results similar to those of Kearns, Mansour and Ng [7], but with "sample complexity" bounds that have only a polynomial rather than exponential dependence on the horizon time. Our method appl...
Keyphrases
policy search method large mdps policy search observable markov decision process method appl sample complexity value estimate following observation equivalent pomdp deterministic transition current state natural way state transition new approach exponential dependence theoretical result general problem horizon time markov decision process
