Abstract

We propose and investigate a method for hierarchical learning and planning in partially observable environments using the framework of Hierarchical Hidden Markov Models (HHMMs). Our main goal is to use hierarchical modeling as a basis for exploring more efficient learning and planning algorithms. As a case study we focus on indoor robot navigation problem and will show how this framework can be used to learn a hierarchy of maps of the environment at different levels of spatial abstraction. We train different families of HHMMs for a real corridor environment and compare them with the standard HMM representation of the same environment. We find significant bene ts to using HHMMs in terms of the fit of the model to the training data, localization of the robot, and the ability to infer the structure of the environment. We also introduce the idea of model reuse that can be used to combine already learned models into a larger model. We describe an extension of the HHMM mode...

Citations