Abstract

When a reinforcement learning agent's next course of action depends on information that is hidden from the sensors because of problems such as occlusion, restricted range, bounded field of view and limited attention, we say the agent suffers from the hidden state problem. State identification techniques use history information to uncover hidden state. Previous approaches to encoding history include: finite state machines [ Chrisman, 1992; McCallum, 1992a ] , recurrent neural networks [ Lin and Mitchell, 1992 ] and genetic programming with indexed memory [ Teller, 1994 ] . A chief disadvantage of all these techniques is their long training time. This report presents instance-based state identification, a new approach to reinforcement learning with state identification that learns with much fewer training steps. Noting that learning with history and learning in continuous spaces both share the property that they begin without knowing the granularity of the state space, the approach appli...

Citations