Research in animal learning and behavioral neuroscience has distinguished between two forms of action control: a habit-based form, which relies on stored action values, and a goal-directed form, which forecasts and compares action outcomes based on a model of the environment. While habit-based control has been the subject of extensive computational research, the computational principles underlying goal-directed control in animals have so far received less attention. In the present paper, we advance a computational framework for goal-directed control in animals and humans. We take three empirically motivated points as founding premises: (1) Neurons in dorsolateral prefrontal cortex represent action policies, (2) Neurons in orbitofrontal cortex represent rewards, and (3) Neural computation, across domains, can be appropriately understood as performing structured probabilistic inference. On a purely computational level, the resulting account relates closely to previous work using Bayesian

& A central question in behavioral science is how we select among choice alternatives to obtain consistently the most beneficial outcomes. Three variables are particularly important when making a decision: the potential payoff, the probability of success, and the cost in terms of time and effort. A key brain region in decision making is the frontal cortex as damage here impairs the ability to make optimal choices across a range of decision types. We simultaneously recorded the activity of multiple single neurons in the frontal cortex while subjects made choices involving the three aforementioned decision variables. This enabled us to contrast the relative contribution of the anterior cingulate cortex (ACC), the orbito-frontal cortex, and the lateral prefrontal cortex to the decision-making process. Neurons in all three areas encoded value relating to choices involving probability, payoff, or cost manipulations. However, the most significant signals were in the ACC, where neurons encoded multiplexed representations of the three different decision variables. This supports the notion that the ACC is an important component of the neural circuitry underlying optimal decision making. &

Tranel, Joseph Wang), to skeptics for forcing us to think harder and write more clearly about the enterprise, and to many neuroscientists (especially John Allman, Paul Glimcher, John O’Doherty and Read Montague) for tutoring and advice over the last few Neuroeconomics seeks to ground microeconomic theory in details about how the brain works (see Zak, 2004; Camerer, Loewenstein and Prelec, 2005; Chorvat and McCabe, 2005; and Sanfey et al, 2006). Neuroeconomics is a subfield of behavioral economics (behavioral economics uses empirical evidence of limits on computation,

#B offered Figure S1. The x-axis and y-axis represent, respectively, the quantities of juices B and A offered in any given trial, and different points in the plane correspond to different offer types. Red dots represent offer types typically employed in our experiments, and the black curve is a hypothetical indifference curve. The indifference curve is non-linear, but close-tolinear within small intervals. In principle, estimating the indifference curve would require testing many different offer types (blues circles). Assuming that the indifference curve is close to linear within small ranges of juice quantity, we can estimate the relative value of the two juices (i.e., the slope of the indifference curve) from a small subset of offer types (red dots).

in the macaque orbitofrontal cortex code relative

An essential component of goal-directed choice is the assignment of values to the different options under consideration. This computation needs to be done both when choosing among appetitive options (e.g., two meals), and when choosing among aversive items (e.g., two undesirable risks). Although much is known about the neural basis of valuation in the appetitive case, little is known about how the brain encodes aversive goal values at the time of decisionmaking. We investigated this question by scanning subjects ’ brains using fMRI while they placed real bids in an economic auction for the right to avoid eating disliked foods. We found that activity in the medial orbitofrontal cortex, the dorsolateral prefrontal cortex, and the insular correlated negatively with aversive goal values (as measured by the bids). Furthermore, a comparison of our results with previous work on the appetitive domain suggests that there is a common valuation system that includes the medial orbitofrontal cortex and the dorsolateral prefrontral cortex that encodes for both appetitive goal values (through increased activation) and for aversive goal values (through decreased activation).

Ventromedial prefrontal cortex (VMF) is thought to be important in human decision making, but studies to date have focused on decision making under conditions of uncertainty, including risky or ambiguous decisions. Other lines of evidence suggest that this area of the brain represents quite basic information about the relative ‘‘economic’ ’ value of options, predicting a role for this region in value-based decision making even in the absence of uncertainty. We tested this prediction in human subjects with VMF damage. Preference judgment is a simple form of value-based decision making under certainty. We asked whether VMF damage in humans would lead to inconsistent preference judgments in a simple pairwise choice task. Twenty-one participants with focal damage to the frontal lobes were compared with 19 age- and educationmatched control subjects. Subjects with VMF damage were significantly more inconsistent in their preferences than controls, whereas those with frontal damage that spared the VMF performed normally. These results argue that VMF plays a necessary role in certain as well as uncertain decision making in humans.

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal VAlues Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra

Abstract not found

Risky decision-making is altered in humans and animals with damage to the orbitofrontal cortex. However, the cellular function of the intact orbitofrontal cortex in processing information relevant for risky decisions is unknown. We recorded responses of single orbitofrontal neurons while monkeys viewed visual cues representing the key decision parameters, reward risk and value. Risk was defined as the mathematical variance of binary symmetric probability distributions of reward magnitudes; value was defined as nonrisky reward magnitude. Monkeys displayed graded behavioral preferences for risky outcomes, as they did for value. A population of orbitofrontal neurons showed a distinctive risk signal: their cues and reward responses covaried monotonically with the variance of the different reward distributions without monotonically coding reward value. Furthermore, a small but statistically significant fraction of risk responses also coded reward value. These risk signals may provide physiological correlates for the role of the orbitofrontal cortex in risk processing.