Reinforcement Learning in Non-Markov Environments

Reinforcement Learning in Non-Markov Environments (1992)

Cached

Download Links

[ftp.gte.com]

by Steven D. Whitehead , Long Ji Lin

Venue:	Artificial Intelligence. Submitted
Citations:	4 - 0 self

BibTeX

@ARTICLE{Whitehead92reinforcementlearning,
    author = {Steven D. Whitehead and Long Ji Lin},
    title = {Reinforcement Learning in Non-Markov Environments},
    journal = {Artificial Intelligence. Submitted},
    year = {1992},
    volume = {8},
    pages = {3--4}
}

Bookmark

OpenURL

Abstract

Recently, techniques based on reinforcement learning (RL) have been used to build systems that learn to perform non-trivial sequential decision tasks. To date, most of this work has focussed on learning tasks that can be described as Markov decision processes (MDPs). While MDPs are useful for modeling a wide range of control problems, there are important problems that are inherently non-Markov. We refer to these as hidden state tasks since they arise when information relevant to identifying the state of the environment is hidden (or missing) from the agent's internal representation. Two important types of control problems that resist Markov modeling are those in which 1) the system has a high degree of control over the information collected by its sensors (e.g., as in active-vision), or 2) the system has a limited set of sensors that do not always provide adequate information about the current state of the environment. Not surprisingly, traditional RL algorithms, which are based primar...

Citations

2888	Induction of Decision Trees - Quinlan - 1986
2307	Learning internal representations by error propagation - Rumelhart, Hinton, et al. - 1986
1965	Dynamic Programming - Bellman - 1957
1314	Finding structure in time - Elman - 1990
1135	Learning from delayed rewards - Watkins - 1989
1060	Learning to predict by the methods of temporal differences - Sutton - 1988
545	Some Studies in Machine Learning using the Game of Checkers - Samuel - 2000
467	Animate vision - Ballard - 1991
427	Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming - Sutton - 1990
334	Practical issues in temporal difference learning - Tesauro - 1992
331	Dynamic Programming: Deterministic and Stochastic Models - Bertsekas - 1987
316	Automatic programming of behavior-based robots using reinforcement learning - Mahadevan, Connell - 1992
314	Eye Movements and Vision - YARBUS - 1967
280	Learning in embedded systems - Kaelbling - 1993
258	Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems - Holland - 1986
255	Self-improving reactive agents based on reinforcement learning, planning, and teaching - Lin - 1992
219	Temporal Credit Assignment in Reinforcement Learning - Sutton - 1984
210	Visual routines - Ullman - 1984
196	Introduction to Stochastic Dynamic Programming - Ross - 1983
173	Reinforcement learning with perceptual aliasing: The perceptual distinctions approach - Chrisman - 1992
157	Neuronlike elements that can solve difficult learning control problems - Barto, Sutton, et al. - 1983
120	The Dynamic Structures of Everyday Life - Agre - 1988
113	Efficient exploration in reinforcement learning - Thrun - 1992
111	BOXES: An experiment in adaptive control - Michie, Chambers - 1968
105	Realtime learning and control using asynchronous dynamic programmming - Baro, Bradtke, et al. - 1991
80	Learning to perceive and act by trial and error - Whitehead, Ballard - 1991
76	Variable resolution dynamic programming: Efficiently learning action maps in multivariate real-valued state-spaces - Moore - 1991
75	Modular construction of time-delay neural networksfor speech recognition - Waibel - 1989
71	Active perception and reinforcement learning - Whitehead, Ballard
61	Programming robots using reinforcement learning and teaching - Lin - 1991
60	Reinforcement Learning for the adaptive control of perception and action - Whitehead - 1992
59	Memory approaches to reinforcement learning in non-Markovian domains - Lin, Mitchell - 1999
53	A teaching method for reinforcement learning - Clouse, Utgoff - 1992
47	Reinforcement learning in Markovian and nonMarkovian environments - Schmidhuber - 1991
38	Triggered Rule Discovery in Classifier Systems - Booker - 1989
32	Self-improving reactive agents:: Case studies of reinforcement learning frameworks - Lin - 1990
32	Theory of neural-analog reinforcement systems and its application to the brain-model problem - Minsky - 1954
28	A study of cooperative mechanisms for faster reinforcement learning - Whitehead - 1991
24	Learning from delayed reinforcement in a complex domain - Chapman, Kaelbling - 1990
24	Learning multiple goal behavior via task decomposition and dynamic policy merging - Whitehead, Karlsson, et al. - 1993
22	Reinforcement learning in connectionist networks: A mathematical analysis - Williams - 1986
18	Statistical Methods. Iowa State - SNEDECOR - 1956
18	Reinforcement-learning connectionist systems - Williams - 1987
14	Transfer of learning across compositions of sequential tasks - Singh - 1991
14	Cost-Sensitive Robot Learning - Tan - 1991
13	Amit Bandyopadhyay. Active Vision - Aloimonos, Weiss - 1988
12	Cost-sensitive reinforcement learning for adaptive classification and control - Tan - 1991
11	Connectionist Modeling and Control of Finite State Environments - Bachrach - 1992
10	Planning with an adaptive world model - Thrun, Moller, et al. - 1991
5	Sensing strategies - Bajcsy, Allen - 1984