Hierarchical Methods for Planning under Uncertainty

Hierarchical Methods for Planning under Uncertainty (2001)

Cached

Download Links

by Joelle Pineau

BibTeX

@MISC{Pineau01hierarchicalmethods,
    author = {Joelle Pineau},
    title = {Hierarchical Methods for Planning under Uncertainty},
    year = {2001}
}

Bookmark

OpenURL

Abstract

The successful integration of robots in people's living environments strongly depends on robots' ability to effectively and meaningfully interact with humans. Human-robot communication requires that a robot be able to select appropriate actions in response to human speech or gestures. This role is played by the robot-based dialogue management system. Designing a dialogue management system is difficult primarily due to the rich multi-modal dimension of human communication, and the uncertainty inherent to the interaction. Uncertainty comes from many sources, including noise in the robot's sensors, changes in the environment, and the ambiguity of meaning in human communication. Furthermore, any model of the human-robot interaction upon which the dialogue management system is based is bound to be incomplete. Probabilistic representations have evolved as the formalism of choice to accommodate stochasticity and uncertainty in AI systems. Partially observable Markov Decision Processes (POMDPs) specifically provide a useful framework for decision-making in the presence of uncertainty, and as such are well suited to address the dialogue management problem. However finding an optimal POMDP policy is computationally infeasible for large problems. There is a strong belief...

Citations

3116	A tutorial on hidden Markov models and selected applications in speech recognition - Rabiner - 1989
2959	Artificial Intelligence: a Modern Approach - Russell, Norvig - 1995
2827	Reinforcement Learning: An Introduction - Sutton, Barto - 1998
629	Planning and acting in partially observable stochastic domains - Kaelbling, Littman, et al. - 1998
342	Decision-Theoretic Planning: Structural Assumptions and Computational Leverage - Boutilier, Dean, et al. - 1999
307	Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition - Dietterich - 1999
285	The Optimal Control of Partially Observable Markov Processes - Sondik - 1971
243	Reinforcement Learning with Selective Perception and Hidden State - McCallum - 1996
235	The optimal control of partially observable Markov processes over a finite horizon - Smallwood, Sondik - 1973
228	Tractable inference for complex stochastic processes - Boyen, Koller - 1998
222	Motivated reinforcement learning - Dayan - 2002
212	Reinforcement learning with hierarchies of machines - Parr, Russell - 1998
176	The Hierarchical Hidden Markov Model: Analysis and Applications - Fine, Singer, et al. - 1998
168	PEGASUS: A policy search method for large MDPs and POMDPs - Ng, Jordan - 2000
165	Survey of partially observable markov decision processes: Theory, models, and algorithms - Monahan - 1982
158	Algorithms for sequential decision making - Littman - 1996
152	Dervish: An office-navigating robot - Nourbakhsh, Powers, et al. - 1995
144	Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes - Cassandra, Littman, et al. - 1997
114	Approximating optimal policies for partially observable stochastic domains - Parr, Russell - 1995
105	Value-function approximations for partially observable markov decision processes - Hauskrecht - 2000
103	Decomposition techniques for planning in stochastic domains - Dean, Lin - 1995
97	Approximate planning in large POMDPs via reusable trajectories - Kearns, Mansour, et al. - 2000
94	Computing optimal policies for partially observable decision processes using compact representations - Boutilier, Poole - 1996
87	Solving POMDPs by search in policy space - Hansen - 1998
84	Monte Carlo POMDPs - Thrun - 2000
77	Probabilistic navigation in partially observable environments - Simmons, Koenig - 1995
72	Efficient algorithms for speech recognition - Ravishankar - 1996
71	Unsupervised learning of probabilistic models for robot navigation - Koenig, Simmons - 1996
70	Solving very large weakly coupled Markov decision processes - Meuleau, Hauskrecht, et al. - 1998
63	Coastal navigation with mobile robot - Roy, Thrun - 2000
61	Reinforcement learning in pomdp’s via direct gradient ascent, in - Baxter, Bartlett - 2000
57	Reinforcement learning for spoken dialogue systems - Singh, Kearns, et al. - 1999
55	How to dynamically merge markov decision processes - Singh, Cohn - 1998
47	An application of reinforcement learning to dialogue strategy selection in a spoken dialogue system for email - Walker - 2000
47	Learning optimal dialogue strategies: A case study of a spoken dialogue agent for email - Walker, Fromer, et al. - 1998
46	Dynamic programming for POMDPs using a factored state representation - Hansen, Feng - 2000
41	Natural Communication with Robots - Torrance - 1994
34	Approximate dynamic programming for sensor management - Castañon - 1997
34	Learning hierarchical partially observable markov decision processs for robot navigation - Theocharous, Rohanimanesh, et al. - 2001
32	Prioritized goal decomposition of Markov decision processes: Toward a synthesis of classical and decision theoretic planning - Boutilier, Brafman, et al. - 1997
32	Active gesture recognition using partial observable Markov decision processes - Darrell, Pentland - 1996
28	Dialogue Strategies Guiding Users to their Communicative Goals - Denecke, Waibel - 1997
23	Spoken dialog management for robots - Roy, Pineau, et al. - 2000
23	Hierarchical optimization of policy-coupled semiMarkov decision processes - Wang, Mahadevan - 1999
21	Value-directed belief state approximation for POMDPs - Poupart, Boutilier - 2000
19	State abstraction in MAXQ hierarchical reinforcement learning - Dietterich
16	Dervish: An oce-navigating robot - Nourbakhsh, Powers, et al. - 1995
12	Hierarchical memory-based reinforcement learning - Hernandez-Gardiol, Mahadevan - 2001
11	Planning medical therapy using partially observable Markov decision processes - Hauskrecht, Fraser - 1998
10	Hierarchical optimal control of MDPs - McGovern, Precup, et al. - 1998