this article should be addressed to Robert Goldstone, Psychology Department, Indiana University, Bloomington, Indiana 47405. Electronic mail may be sent to rgoldsto @ ucs.indiana.edu

Two connectionist models of attention in associative learning, previously used to model human category learning, are shown to have special cases that are essentially equivalent to N. J. Mackintosh's (1975, Psychological Review, 82, 276 298) classic model of attention in animal learning. The models unify formulas for associative weight change with formulas for attentional change, under a common goal of error reduction. Error-driven attentional shifting accelerates learning of new associations but also protects previously learned associations from retroactive interference. The models are fit to data from a recent experiment in human associative learning (J. K. Kruschke 6 N. J. Blair, 2000, Psychonomic Bulletin 6 Review, 7, 636 645), which shows that blocking of learning involves learned inattention. The approach also provides a novel and unifying theory of latent inhibition (the preexposure effect) in terms of blocking. The discussion summarizes how the approach accounts for a variety of other ``irrational' ' phenomena in associative learning, including base rate effects, perseveration of attention through relevance

Shaping by successive approximations is an important animal training technique in which behavior is gradually adjusted in response to strategically timed reinforcements. We describe a computational model of this shaping process and its implementation on a mobile robot. Innate behaviors in our model are sequences of actions and enabling conditions, and shaping is a behavior editing process realized by multiple editing mechanisms. The model replicates some fundamental phenomena associated with instrumental learning in animals, and allows an RWI B21 robot to learn several distinct tasks derived from the same innate behavior. 1. Introduction Service dogs trained to assist a disabled person will respond to over 60 verbal commands to, for example, turn on lights, open a refrigerator door, or retrieve a dropped object [9]. Chicks can be taught to play a toy piano (peck out a key sequence until a reinforcement is received at the end of the tune) [6], and rats have been conditioned to perform c...

Some recent formal models of Pavlovian and instrumental conditioning contain internal paradoxes that restrict their predictive power. These paradoxes can be traced to an inadequate formulation of how mechanisms of short-term memory and long-term memory work together to control the shifting balance between the processing of expected and unexpected events. Once this formulation is strengthened, a unified processing framework is suggested wherein attentional and orienting subsystems coexist in a complementary relationship that controls the adaptive self-organization of internal representations in response to expected and unexpected events. In this framework, conditioning and attentional constructs can be more directly validated by interdisciplinary paradigms in which seemingly disparate phenomena can be shown to share similar physiological and pharmacological mechanisms. A model of cholinergic-catecholaminergic interactions suggests how drive, reinforcer, and arousal inputs regulate motivational baseline, hysteresis, and rebound, with the hippocampus as a final common path. Extinction, conditioned emotional responses, conditioned avoidance responses, secondary

Hippocampal participation in classical conditioning is described in terms of a multilayer network that portrays stimulus configuration. The network (a) describes behavior in real time, (b) incorporates a layer of "hidden " units positioned between input and output units, (c) includes inputs that are connected to the output directly as well as indirectly through the hidden-unit layer, and (d) uses a biologically plausible backpropagation procedure to train the hidden-unit layer. Nodes and connections in the neural network are mapped onto regional cerebellar, cortical, and hippocampal circuits, and the effect of lesions of different brain regions is formally studied. Computer simulations of the following classical conditioning paradigms are presented: acquisition of delay and trace conditioning, extinction, acquisition-extinction series of delay conditioning, blocking, overshadowing, discrimination acquisition, discrimination reversal, feature-positive discrimination, conditioned inhibition, negative patterning, positive patterning, and generalization. The model correctly describes the effect of hippocampal and cortical lesions in many of these paradigms, as well as neural activity in hippocampus and medial septum during classical conditioning. Some of these results might be extended to the description of anterograde amnesia in human patients. In spite of the vast amount of behavioral and physiological

John P. O’Doherty, V(tUCS) and V(tUCS � 1) generates a positive prediction error that, in the simplest form of TD learning, is used to increment the value at time tUCS – 1 (in proportion to

Recent research on contingency judgment indicates that the judged predictiveness of a cue is dependent on the predictive strengths of other cues. Two classes of models correctly predict such cue interaction: associative models and statistical models. However, these models differ in their predictions about the effect of trial order on cue interaction. In five experiments reported here, college students viewed trial-by-trial data regarding several medical symptoms and a disease, judging the predictive strength of each symptom with respect to the disease. The results indicate that trial order influences the manner in which cues interact, but that neither the associative nor the statistical models can fully account for the data pattern. A possible variation of an associative account is discussed. The ability to detect predictive relationships among envi-ronmental events grants humans and other animals a distinct benefit. Therefore, the mechanisms underlying this ability are of considerable interest. Recent research with humans on judgments of contingencies has shed light on these mecha-nisms. It has suggested two classes of theoretical models that

C. R. Gallistel and J. Gibbon (2000) presented quantitative data on the speed with which animals acquire behavioral responses during autoshaping, together with a statistical model of learning intended to account for them. Although this model captures the form of the dependencies among critical variables, its detailed predictions are substantially at variance with the data. In the present article, further key data on the speed of acquisition are used to motivate an alternative model of learning, in which animals can be interpreted as paying different amounts of attention to stimuli according to estimates of their differential reliabilities as predictors. In autoshaping experiments on pigeons, birds acquire a classically conditioned peck response to a lighted key associated, irrespective of their actions, with the delivery of food (Brown & Jenkins, 1968). As stressed persuasively by Gallistel and Gibbon (2000), there is substantial experimental evidence in favor of a simple quantitative relationship between the speed of acquisition in autoshaping and the three critical variables shown in Figure 1A. The first is I, the length of intertrial interval; the second is T, the time during the trial for which the conditioned stimulus (CS; a light in this case) is presented; and the third is the training schedule, 1/S, which is the fractional number of deliveries per light (for those birds that were only partially reinforced). Here, acquisition speeds are typically measured in terms of the number of trials it takes until a certain behavioral criterion is met, such as pecking during the time the light is illuminated on three out of four

Present neural models of classical conditioning all suffer from the same shortcoming: local representation of information (therefore, very precise neural prewiring is necessary). As an alternative we develop two neural models of classical conditioning which rely on distributed representations of information. Both models are of the Hopfield type. In the first model the existence of transmission delays is used to store temporal relations. The second model is based on interactions between spatially separated neural fields. Using tools from statistical mechanics we show that behavioural constraints can be met only if the Hebb rule is extended with inter- or intrasynaptic competition. 2 3 1. Introduction Connectionism has redirected the attention of cognitive scientists to learning and to the neural substrate in which cognitive processes are implemented. Conditioning has become an important field in which ideas from neural networks, behavioural science and neurophysiology are combined. ...

Is attention a purely perceptual process or is it in any way related to motor control? The aim of this article is to show that attention puts similar demands on a cognitive system as motor control and present evidence supporting the view that similar mechanisms operate in the two processes. A computational model of attention is presented that uses habituation as well as classical and instrumental conditioning to explain a number of attentional processes. Evidence from neurophysiology is reviewed that suggest that attention is controlled in a way similar to actions. This view makes it possible to adapt traditional learning theoretical mechanisms to the control of attention. Computer simulations are presented that illustrates the operation of the model.