Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density

Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density (2001)

Cached

Download Links

[www.lunabots.com]
[idea.cs.ou.edu]
[www.cs.ou.edu]

Other Repositories/Bibliography

DBLP

by Amy Mcgovern , Andrew G. Barto

Venue:	In Proceedings of the eighteenth international conference on machine learning
Citations:	93 - 16 self

BibTeX

@INPROCEEDINGS{Mcgovern01automaticdiscovery,
    author = {Amy Mcgovern and Andrew G. Barto},
    title = {Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density},
    booktitle = {In Proceedings of the eighteenth international conference on machine learning},
    year = {2001},
    pages = {361--368},
    publisher = {Morgan Kaufmann}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

This paper presents a method by which a reinforcement learning agent can automatically discover certain types of subgoals online. By creating useful new subgoals while learning, the agent is able to accelerate learning on the current task and to transfer its expertise to other, related tasks through the reuse of its ability to attain subgoals. The agent discovers subgoals based on commonalities across multiple paths to a solution. We cast the task of finding these commonalities as a multiple-instance learning problem and use the concept of diverse density to find solutions. We illustrate this approach using several gridworld tasks. 1.

Citations

2827	Reinforcement Learning: An Introduction - Sutton, Barto - 1998
342	Between MDPS and semi-MDPS: a framework for temporal abstraction in reinforcement learning - Sutton, Precup, et al. - 1999
255	Self-improving reactive agents based on reinforcement learning, planning, and teaching - Lin - 1992
180	Solving the multipleinstance problem with axis-parallel rectangles - Dietterich, Lathrop, et al. - 1997
137	A framework for multiple-instance learning - Maron, Lozano-Pérez - 1998
131	Macro operators: A weak method for learning - Korf - 1985
96	On representations of problems of reasoning about actions - Amarel - 1968
94	A heuristic approach to the discovery of macro-operators - Iba - 1989
78	The theory of learning by doing - Simon - 1979
51	Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments - Digney - 1996
43	Learning hierarchical control structures for multiple tasks and changing environments - Digney - 1998
38	Roles of macro-actions in accelerating reinforcement learning - McGovern, Sutton, et al. - 1997
17	Composing functions to speed up reinforcement learning in a changing world - Drummond - 1998
15	Reusing old policies to accelerate learning on new MDPs - Bernstein - 1999
11	Learning from Ambiguity. Doctoral dissertation - Maron - 1998
9	acQuire-macros: An algorithm for automatically learning macro-actions - McGovern - 1998
9	Temporal abstraction in reinforcement learning. Doctoral dissertation - Precup - 2000