Causal Graph Based Decomposition of Factored MDPs

Causal Graph Based Decomposition of Factored MDPs (2006)

Cached

Download Links

by Anders Jonsson , Andrew Barto

Venue:	JOURNAL OF MACHINE LEARNING RESEARCH
Citations:	18 - 3 self

BibTeX

@ARTICLE{Jonsson06causalgraph,
    author = {Anders Jonsson and Andrew Barto},
    title = { Causal Graph Based Decomposition of Factored MDPs},
    journal = {JOURNAL OF MACHINE LEARNING RESEARCH},
    year = {2006},
    volume = {7},
    pages = {2259--2301}
}

Bookmark

OpenURL

Abstract

We present Variable Influence Structure Analysis, or VISA, an algorithm that performs hierarchical decomposition of factored Markov decision processes. VISA uses a dynamic Bayesian network model of actions, and constructs a causal graph that captures relationships between state variables. In tasks

Citations

2827	Reinforcement Learning: An Introduction - Sutton, Barto - 1998
1962	Statecharts: A visual formalism for complex systems - Harel - 1987
1555	STRIPS: A new approach to the application of theorem proving to problem solving - Fikes, Nilsson - 1971
498	Markov Decision Processes - Puterman - 1994
472	Learning to act using real-time dynamic programming - Barto, Bradtke, et al. - 1995
396	A model for reasoning about persistence and causation - Dean, Kanazawa - 1990
342	Between MDPS and semi-MDPS: a framework for temporal abstraction in reinforcement learning - Sutton, Precup, et al. - 1999
212	Reinforcement learning with hierarchies of machines - Parr, Russell - 1998
200	Exploiting structure in policy construction - Boutilier, Dearden, et al. - 1995
146	SPUDD: Stochastic planning using decision diagrams - Hoey, St-Aubin, et al. - 1999
103	Decomposition techniques for planning in stochastic domains - Dean, Lin - 1995
99	Reinforcement learning methods for continuous-time Markov decision problems - Bradtke, Duff - 1995
98	Hierarchical Control and Learning for Markov Decision Processes - Parr - 1998
98	Finding structure in reinforcement learning - Thrun, Schwartz - 1995
97	R.: Model minimization in markov decision processes - Dean, Givan - 1997
93	Automatic discovery of subgoals in reinforcement learning using diverse density - McGovern, Barto - 2001
70	A planning heuristic based on causal graph analysis - Helmert
69	A markov decision process - Bellman - 1957
65	Discovering hierarchy in reinforcement learning with HEXQ - Hengst
58	Max-norm projections for factored MDPs - Guestrin, Koller, et al. - 2001
56	Efficient reinforcement learning in factored MDPs - Kearns, Koller - 1999
51	Using relative novelty to identify useful temporal abstractions in reinforcement learning - S¸im¸sek, Barto - 2004
51	Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments - Digney - 1996
46	Symbolic heuristic search for factored Markov decision processes, in: AAAI’02 - Feng, Hansen - 2002
32	Identifying useful subgoals in reinforcement learning by local graph partitioning - Simsek, Wolfe, et al. - 2005
29	Dynamic abstraction in reinforcement learning via clustering - Mannor, Menache, et al. - 2004
28	An Algebraic Approach to Abstraction in Reinforcement Learning - Ravindran - 2004
25	Active learning of causal bayes net structure - Murphy
18	Continuous-time hierarchical reinforcement learning - Ghavamzadeh, Mahadevan - 2001
13	Symbolic generalization for on-line planning - Feng, Hansen, et al. - 2003
12	A causal approach to hierarchical decomposition of factored MDPs - Jonsson, Barto - 2005
11	Unsupervised active learning in large domains - Steck, Jaakkola - 2002
4	Q-Cut – Dynamic Discovery of Sub-Goals - Menache, Mannor, et al. - 2002
4	PolicyBlocks: An Algorithm for Creating Useful - Pickett, Barto
2	State Abstraction - Dietterich