Model Reduction Techniques for Computing Approximately Optimal Solutions for Markov Decision Processes (1997)
| Venue: | In Thirteenth Conference on Uncertainty in Artificial Intelligence |
| Citations: | 48 - 7 self |
BibTeX
@INPROCEEDINGS{Dean97modelreduction,
author = {Thomas Dean and Robert Givan and Sonia Leach},
title = {Model Reduction Techniques for Computing Approximately Optimal Solutions for Markov Decision Processes},
booktitle = {In Thirteenth Conference on Uncertainty in Artificial Intelligence},
year = {1997},
pages = {124--131},
publisher = {Morgan Kaufmann}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces. We introduce a property of state space partitions which we call ffl-homogeneity. Intuitively, an ffl-homogeneous partition groups together states that behave approximately the same under all or some subset of policies. Borrowing from recent work on model minimization in computer-aided software verification, we present an algorithm that takes a factored representation of an MDP and an 0 ffl 1 and computes a factored ffl-homogeneous partition of the state space. This partition defines a family of related MDPs---those MDP's with state space equal to the blocks of the partition, and transition probabilities "approximately" like those of any (original MDP) state in the source block. To formally study such families of MDPs, we introduce the new notion of a "bounded parameter MDP" (BMDP), which is a family of (traditional) MDPs defined by specifying upper...
