Results 1 
5 of
5
Exploiting redundancy for flexible behavior: Unsupervised learning in a modular sensorimotor control architecture
 Psychological Review
"... Autonomously developing organisms face several challenges when learning reaching movements. First, motor control is learned unsupervised or selfsupervised. Second, knowledge of sensorimotor contingencies is acquired in contexts in which action consequences unfold in time. Third, motor redundancies ..."
Abstract

Cited by 13 (8 self)
 Add to MetaCart
(Show Context)
Autonomously developing organisms face several challenges when learning reaching movements. First, motor control is learned unsupervised or selfsupervised. Second, knowledge of sensorimotor contingencies is acquired in contexts in which action consequences unfold in time. Third, motor redundancies must be resolved. To solve all 3 of these problems, the authors propose a sensorimotor, unsupervised, redundancyresolving control architecture (SURE_REACH), based on the ideomotor principle. Given a 3degreesoffreedom arm in a 2dimensional environment, SURE_REACH encodes 2 spatial arm representations with neural population codes: a hand endpoint coordinate space and an angular arm posture space. A posture memory solves the inverse kinematics problem by associating hand endpoint neurons with neurons in posture space. An inverse sensorimotor model associates posture neurons with each other actiondependently. Together, population encoding, redundant posture memory, and the inverse sensorimotor model enable SURE_REACH to learn and represent sensorimotor grounded distance measures and to use dynamic programming to reach goals efficiently. The architecture not only solves the redundancy problem but also increases goal reaching flexibility, accounting for additional task constraints or realizing obstacle avoidance. While the spatial population codes resemble neurophysiological structures, the simulations confirm the flexibility and plausibility of the model by mimicking previously published data in armreaching tasks.
An Algorithmic Description of ACS2
, 2002
"... The various modi cations and extensions of the anticipatory classi er system (ACS) recently led to the introduction of ACS2, an enhanced and modi ed version of ACS. This chapter provides an overview over the system including all parameters as well as framework, structure, and environmental in ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
The various modi cations and extensions of the anticipatory classi er system (ACS) recently led to the introduction of ACS2, an enhanced and modi ed version of ACS. This chapter provides an overview over the system including all parameters as well as framework, structure, and environmental interaction. Moreover, a precise description of all algorithms in ACS2 is provided.
Designing Efficient Exploration with MACS: Modules and Function Approximation
 In Proceedings of the Genetic and Evolutionary Computation Conference 2003 (GECCO03
, 1993
"... MACS (Modular Anticipatory Classifier System) is a new Anticipatory Classifier System. With respect to its predecessors, ACS, ACS2 and YACS, the latent learning process in MACS is able to take advantage of new regularities. Instead of anticipating all attributes of the perceived situations in the sa ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
MACS (Modular Anticipatory Classifier System) is a new Anticipatory Classifier System. With respect to its predecessors, ACS, ACS2 and YACS, the latent learning process in MACS is able to take advantage of new regularities. Instead of anticipating all attributes of the perceived situations in the same classifier, MACS only anticipates one attribute per classifier. In this paper we describe how the model of the environment represented by the classifiers can be used to perform active exploration and how this exploration policy is aggregated with the exploitation policy. The architecture is validated experimentally. Then we draw more general principles from the architectural choices giving rise to MACS. We show that building a model of the environment can be seen as a function approximation problem which can be solved with Anticipatory Classifier Systems such as MACS, but also with accuracybased systems like XCS or XCSF, organized into a Dyna architecture.
Generalized State . . . Anticipatory Learning Classifier System
 IN BUTZ M., SIGAUD O., GÉRARD P. (EDS.), ANTICIPATORY BEHAVIOR IN
, 2002
"... This paper introduces generalized state values to the anticipatory learning classifier system ACS2. Previous studies with ACS2 showed that the system reliably evolves a generalized predictive model in typical Markov decision process (MDP). The predictive model approximates the state transition funct ..."
Abstract
 Add to MetaCart
This paper introduces generalized state values to the anticipatory learning classifier system ACS2. Previous studies with ACS2 showed that the system reliably evolves a generalized predictive model in typical Markov decision process (MDP). The predictive model approximates the state transition function of the MDP in a compact, generalized form. However, it was also shown that the evolving predictive model might be overgeneral for an accurate representation of reinforcement values. Thus, a function approximation module is added that approximates state values. In combination, actual action choice depends on state values predicted by the means of the predictive model yielding anticipatory behavior. It is shown that the function approximation module accurately generalizes the state value function in the investigated MDP. We also
L'apprentissage Par Renforcement Indirect Dans Les Systèmes De Classeurs
"... Les systmes de classeurs sont des systmes base de rgles qui combinent une capacit d'apprentissage par renforcement et une capacit de gnralisation. Au lieu d'associer des valeurs des couples (tat, action), comme c'est le cas dans le cadre de l'apprentissage par renforcement tab ..."
Abstract
 Add to MetaCart
Les systmes de classeurs sont des systmes base de rgles qui combinent une capacit d'apprentissage par renforcement et une capacit de gnralisation. Au lieu d'associer des valeurs des couples (tat, action), comme c'est le cas dans le cadre de l'apprentissage par renforcement tabulaire, ils associent des valeurs des couples (condition, action), dans lesquels la partie condition peut tre vrie par plusieurs tats, ce qui permet la gnralisation.