Shaping by successive approximations is an important animal training technique in which behavior is gradually adjusted in response to strategically timed reinforcements. We describe a computational model of this shaping process and its implementation on a mobile robot. Innate behaviors in our model are sequences of actions and enabling conditions, and shaping is a behavior editing process realized by multiple editing mechanisms. The model replicates some fundamental phenomena associated with instrumental learning in animals, and allows an RWI B21 robot to learn several distinct tasks derived from the same innate behavior. 1. Introduction Service dogs trained to assist a disabled person will respond to over 60 verbal commands to, for example, turn on lights, open a refrigerator door, or retrieve a dropped object [9]. Chicks can be taught to play a toy piano (peck out a key sequence until a reinforcement is received at the end of the tune) [6], and rats have been conditioned to perform c...

Research in animal learning and behavioral neuroscience has distinguished between two forms of action control: a habit-based form, which relies on stored action values, and a goal-directed form, which forecasts and compares action outcomes based on a model of the environment. While habit-based control has been the subject of extensive computational research, the computational principles underlying goal-directed control in animals have so far received less attention. In the present paper, we advance a computational framework for goal-directed control in animals and humans. We take three empirically motivated points as founding premises: (1) Neurons in dorsolateral prefrontal cortex represent action policies, (2) Neurons in orbitofrontal cortex represent rewards, and (3) Neural computation, across domains, can be appropriately understood as performing structured probabilistic inference. On a purely computational level, the resulting account relates closely to previous work using Bayesian

From Reactive Behaviour to Adaptive Behaviour Motivational models for behaviour in animals and robots E. H. Spier, Balliol College, Trinity Term 1997 A thesis submitted for the degree of Doctor of Philosophy This thesis presents one possible way to design a control architecture that can be used to govern artificial animals. Such artefacts perform multiple-tasks and are expected to exist in a somewhat hostile environment -- they have to be adaptive. It also defends the position that automata, and animals, need not use reasoning to perform intelligent behaviour. Drawing from an ethological conception of motivation, a mathematical framework was described, computer simulations performed and preliminary work on a real robot discussed. It was shown that a reactive motivational algorithm performs better than alternatives that use simplistic models of the world, in a multiple resource foraging task. The reactive motivational framework was then extended to encompass instrumental behaviour as ...

In this paper we show that a phenomenon in animal learning theory (the outcome devaluation effect) for which there is dispute over whether explicit representations and symbolic reasoning is required for its performance, does not require such things. This is done using a reactive motivational model, previously inspired from ethological thought, to which some simple reinforcement learning rules are attached. An instantation of the model is used as the control system of an animat in a spatial computer simulation and it succeeds in learning the necessary parameters to allow the behaviour sequencing system to exhibit the phenomenon. 1 Introduction How complex can a reactive animat's behaviours get before some begin to appeal for a return to the well established rational techniques in classical artificial intelligence ? This paper offers an analysis and performance of a phenomenon in animal learning theory that provokes controversy about the type and complexity of the cognitive machinery ...

available at www.sciencedirect.com www.elsevier.com/locate/brainres Research Report Effect on movement selection of an evolving sensory representation: A multiple controller model of skill acquisition

Abstract not found