DMCA
Dynamic Abstraction in Reinforcement Learning via Clustering (2004)
Cached
Download Links
| Venue: | In Proceedings of the Twenty-First International Conference on Machine Learning |
| Citations: | 53 - 2 self |
BibTeX
@INPROCEEDINGS{Mannor04dynamicabstraction,
author = {Shie Mannor and Ishai Menache and Amit Hoze and Uri Klein},
title = {Dynamic Abstraction in Reinforcement Learning via Clustering},
booktitle = {In Proceedings of the Twenty-First International Conference on Machine Learning},
year = {2004},
pages = {560--567},
publisher = {ACM Press}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We consider a graph theoretic approach for automatic construction of options in a dynamic environment. A map of the environment is generated on-line by the learning agent, representing the topological structure of the state transitions. A clustering algorithm is then used to partition the state space to di#erent regions. Policies for reaching the di#erent parts of the space are separately learned and added to the model in a form of options (macro-actions). The options are used for accelerating the Q-Learning algorithm. We extend the basic algorithm and consider building a map that includes preliminary indication of the location of "interesting " regions of the state space, where the value gradient is significant and additional exploration might be beneficial. Experiments indicate significant speedups, especially in the initial learning phase.
Keyphrases
dynamic abstraction reinforcement learning state space di erent part additional exploration learning agent preliminary indication automatic construction value gradient significant speedup initial learning phase state transition dynamic environment clustering algorithm topological structure erent region basic algorithm graph theoretic approach q-learning algorithm
