Abstract

The standard reinforcement learning view of the involvement of neuromodulatory systems in instrumental conditioning includes a rather straightforward conception of motivation as prediction of sum future reward. Competition between actions is based on the motivating characteristics of their consequent states in this sense. Substantial, careful, experiments reviewed in Dickinson & Balleine, into the neurobiology and psychology of motivation shows that this view is incomplete. In many cases, animals are faced with the choice not between many different actions at a given state, but rather whether a single response is worth executing at all. Evidence suggests that the motivational process underlying this choice has different psychological and neural properties from that underlying action choice. We describe and model these motivational systems, and consider the way they interact.

Citations

1135	Learning from delayed rewards - Watkins - 1989
649	Reinforcement learning - Sutton, Barto - 1998
324	PR: A neural substrate of prediction and reward - Schultz, Dayan, et al. - 1997
150	A framework for mesencephalic dopamine systems based on predictive Hebbian learning - PR, Dayan, et al. - 1996
58	NJ: The psychology of animal learning - Mackintosh - 1974
55	TD models: Modeling the world at a mixture of time scales - Sutton - 1995
40	Advantage updating - Baird - 1993
34	The learning of world models by connectionist networks - Sutton, Pinette - 1985
33	Cognition and control in schizophrenia: A computational model of dopamine and prefrontal function - Braver, Barch, et al. - 1999
33	Sejnowski TJ - PR, Dayan - 1996
28	Amygdala circuitry in attentional and representational processes. Trends Cogn Sci 3: 65–73 - PC, Gallagher - 1999
27	Neuronlike elements that can solve di cult learning control problems - Barto, Sutton, et al. - 1983
26	A: Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology - BW, Dickinson - 1998
19	Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning - Schoenbaum, AA, et al. - 1999
18	Integrative activity of the brain; an interdisciplinary approach - Konorski - 1967
16	An adaptive network that constructs and uses an internal model of its world - Sutton, Barto - 1981
15	Variations in the sensitivity of instrumental responding to reinforcer devaluation - Adams - 1982
12	Motivational control after extended instrumental training - Dickinson, Balleine, et al. - 1995
11	Balleine B: Motivational control of goal-directed action - Dickinson - 1994
10	Motivational control of heterogeneous instrumental chains - Balleine, Garner, et al. - 1995
9	From reactive behaviour to adaptive behaviour - Spier - 1997
7	Reward learning: Reinforcement, incentives, and expectations. In D.L. Medin �Ed.), The psychology of learning and motivation �Vol - Berridge - 2001
5	The determiners of behavior at a choice point - Tolman - 1938
4	Palatability shift of a salt-associated incentive during sodium depletion - Berridge, Schulkin - 1989
4	Improving generalisation for temporal dierence learning: The successor representation - Dayan - 1993
4	Discriminative conditioning: I. A discriminative property of conditioned anticipation - Estes - 1943
2	BW (2001) The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell - Corbit, Muir
2	The eect of two ways of devaluing the unconditioned stimulus after rst- and second-order appetitive conditioning - Holland - 1975
1	Natural Intelligence in Articial Creatures - Balkenius - 1995
1	Conditioned Reexes and Neuron Organization - Konorski - 1948