Abstract

A basic classifier system, ZCS, is presented which keeps much of Holland's original framework but simplifies it to increase understandability and performance. ZCS's relation to Q-learning is brought out, and their performances compared in environments of two difficulty levels. Extensions to ZCS are proposed for temporary memory, better action selection, more efficient use of the genetic algorithm, and more general classifier representation. Key words Classifier systems, Q-learning, temporary memory, action selection, restricted mating, s-classifiers, genetic programming 2 1. Introduction A classifier system is a learning system in which a set of condition-action rules called classifiers compete to control the system and gain credit based on the system's receipt of reinforcement from the environment. A classifier's cumulative credit, termed strength, determines its influence in the control competition and in an evolutionary process using a genetic algorithm in which new, plausibly b...

Citations

6071	Genetic Algorithms - Goldberg - 1989
1313	Finding structure in time - Elman - 1990
1135	Learning from delayed rewards - Watkins - 1989
805	Genetic Programming - Koza, Rice - 1992
447	Adaptive switching circuits - Widrow, Ho€ - 1960
316	Automatic programming of behavior-based robots using reinforcement learning - Mahadevan, Connell - 1992
280	Learning in embedded systems - Kaelbling - 1993
258	Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems - Holland - 1986
178	Reinforcement Learning for Robots Using Neural Networks - Lin - 1993
100	Knowledge growth in an artificial animal - Wilson - 1985
92	Reinforcement learning architectures for animats - Sutton - 1992
88	Memoryless policies: Theoretical limitations and practical results. In From Animals to Animats 3 - Littman - 1994
85	A reinforcement learning method for maximizing undiscounted rewards - Schwartz - 1993
68	Classifier Systems and the Animat Problem - Wilson - 1987
64	Intelligent Behavior as an Adaptation to the Task Environment - Booker - 1982
62	The animat path to AI - Wilson - 1991
59	Adaptation - Holland - 1992
38	Triggered Rule Discovery in Classifier Systems - Booker - 1989
36	Reinforcement learning and adaptive critic methods - Barto - 1992
30	Hierarchical Credit Allocation in a Classifier System - Wilson - 1987
27	A New Approach to Fuzzy Classifier System - Parodi, Bonelli - 1993
26	Genetic and non-genetic operators in ALECSYS - Dorigo - 1993
26	The evolution of strategies for multi-agent environments - Grefenstette - 1992
23	An optimization-based categorization of reinforcement learning environments - Littman - 1993
15	1992], ‘Introduction - the challenge of reinforcement learning - Sutton
14	Default Hierarchy Formation and Memory Exploitation in Learning Classifiers Systems - Smith - 1991
12	The Emergence of Coupled Sequences of Classifiers - Riolo - 1989
11	The fuzzy classifier system: A classifier system for continuously varying variables - Valenzuela-Rend��n - 1991
10	Adding "foveal vision" to Wilson's animat - Cliff - 1993
9	Representing Attribute-Based Concepts in a Classifier System - Booker - 1990
8	Dynamic Planning for Classifier Systems - Roberts - 1993
8	Bid competition and specificity reconsidered - Wilson - 1988
1	Q-learning. Talk presented at the workshop "Reinforcement Learning: What We Know, What We Need - Watkins - 1993