Abstract

Markov decision processes (MDPs) are models of dynamic decision making under uncertainty. These models arise in diverse applications and have been developed extensively in fields such as operations research, control engineering, and the decision sciences in general. Recent research, especially in artificial intelligence, has highlighted the significance of studying the computational properties of MDP problems. We address

Citations

7320	Introduction to Algorithms - CORMEN, LEISERSON, et al. - 1985
3376	Introduction to Automata Theory Languages and Computation. 2nd edition. Addison-Wesley Publishing Company, 2000. de - Hopcroft, Ullman - 2004
1965	Dynamic Programming - Bellman - 1957
1950	Computational Complexity - Papadimitriou - 1995
1566	Randomized Algorithms - MOTWANI, RAGHAVAN - 1995
1555	STRIPS: A new approach to the application of theorem proving to problem solving - Fikes, Nilsson - 1971
1134	Reinforcement learning: A survey - Kaelbling, Littman, et al. - 1996
1107	Network Flows: Theory, Algorithms, and Applications - Ahuja, Magnanti, et al. - 1993
1055	Combinatorial Optimization: Algorithms and Complexity - Papadimitriou, Steiglitz - 1982
498	Markov decision processes - Puterman - 1995
342	Dynamic Programming and Optimal Control, Athena Scientific - Bertsekas - 2000
331	Dynamic Programming: Deterministic and Stochastic Models - Bertsekas - 1987
285	The Optimal Control of Partially Observable Markov Processes - Sondik - 1971
253	The complexity of Markov decision processes - Papadimitriou, Tsisiklis - 1987
246	The computational complexity of propositional STRIPS planning - Bylander - 1994
243	Reinforcement Learning with Selective Perception and Hidden State - McCallum - 1996
235	An algorithm for probabilistic planning - Kushmerick, Hanks, et al. - 1995
235	The optimal control of partially observable Markov processes over a finite horizon - Smallwood, Sondik - 1973
224	Applying parallel computation algorithms in design of serial algorithms - Megiddo - 1983
173	A characterization of the minimum cycle mean in a digraph - Karp - 1978
165	Survey of partially observable markov decision processes: Theory, models, and algorithms - Monahan - 1982
158	Algorithms for sequential decision making - Littman - 1996
151	A survey of algorithmic methods for partially observed Markov decision processes - Lovejoy - 1991
133	Decision theoretic planning: Structural assumptions and computational leverage - Boutilier, Dean, et al. - 1999
133	Probabilistic automata - Rabin - 1963
131	Introduction to Probabilistic Automata - Paz - 1971
116	Theory of linear and integer programming. Wiley-Interscience series in discrete mathematics and optimization - Schrijver - 1998
113	Complexity, decidability and undecidability results for domain-independent planning - Erol, Nau, et al. - 1995
111	Exact and approximate algorithms for partially observable Markov decision processes - Cassandra - 1998
100	A polynomial algorithm for linear programming - Khachian - 1979
88	Xavier: A robot navigation architecture based on partially observable markov decision process models - Koenig, Simmons - 1998
87	A subexponential randomized simplex algorithm - Kalai - 1992
82	A survey of computational complexity results in systems and control - Blondel, Tsitsiklis
74	Probabilistic propositional planning: Representations and complexity - Littman - 1997
71	The computational complexity of probabilistic planning - Littman, Goldsmith, et al. - 1998
65	Policy iteration for factored MDPs - Koller, Parr - 2000
53	Finding minimum-cost circulations by cancelling negative cycles - Goldberg, Tarjan
49	On algorithms for simple stochastic games - Condon - 1993
48	Simple and fast algorithms for linear and integer programs with two variables per inequality - Hochbaum, Naor
44	On the complexity of space bounded interactive proofs (extended abstract - Condon, Lipton - 1989
42	The d-step conjecture and its relatives - Klee, Kleinschmidt - 1987
41	Faster maximum and minimum mean cycle algorithms for system performance analysis - Dasdan, Gupta - 1998
41	Dynamic Programming, Models and Applications - Denardo - 1982
37	A Subexponential Randomized Algorithm for the Simple Stochastic Game Problem - Ludwig - 1995
33	A polynomial time algorithm for solving systems of linear inequalities with two variables per inequality - Aspvall, Shiloach - 1980
33	Tight bounds and 2-approximation algorithms for integer programs with two variables per inequality - Hochbaum, Megiddo, et al. - 1993
33	J.B.: Parametric shortest path algorithms with an application to cyclic staffing - Karp, Orlin - 1981
33	Markov Decision Processes - White - 1993
32	Active gesture recognition using partial observable Markov decision processes - Darrell, Pentland - 1996
31	Optimal control of a Markov process with incomplete state information - ˚Aström - 1965