Research in animal learning and behavioral neuroscience has distinguished between two forms of action control: a habit-based form, which relies on stored action values, and a goal-directed form, which forecasts and compares action outcomes based on a model of the environment. While habit-based control has been the subject of extensive computational research, the computational principles underlying goal-directed control in animals have so far received less attention. In the present paper, we advance a computational framework for goal-directed control in animals and humans. We take three empirically motivated points as founding premises: (1) Neurons in dorsolateral prefrontal cortex represent action policies, (2) Neurons in orbitofrontal cortex represent rewards, and (3) Neural computation, across domains, can be appropriately understood as performing structured probabilistic inference. On a purely computational level, the resulting account relates closely to previous work using Bayesian

Abstract not found

We show that ants can reconnoitre their surroundings and in effect plan for the future. Temnothorax albipennis colonies use a sophisticated strategy to select a new nest when the need arises. Initially, we presented colonies with a new nest of lower quality than their current one that they could explore for one week without a need to emigrate. We then introduced a second identical low quality new nest and destroyed their old nest so that they had to emigrate. Colonies showed a highly significant preference for the (low quality) novel new nest over the identical but familiar one. In otherwise identical experiments, colonies showed no such discrimination when the choice was between a familiar and an unfamiliar highquality nest. When, however, either all possible pheromone marks were removed, or landmarks were re-orientated, just before the emigration, the ants chose between identical low-quality new nests at random. These results demonstrate for the first time that ants are capable of assessing and retaining information about the quality of potential new nest sites, probably by using both pheromones and landmark cues, even though this information may only be of strategic value to the colony in the future. They seem capable, therefore, of latent learning and, more explicitly, learning what not to do.

The cognitive abilities of the anticipatory classifier system (ACS) have already been successfully shown in earlier work (Stolzmann et al 2000). This report takes inspiration from some philosophical ideas for the purpose of dreaming in animals and humans during REM sleep. This is supported by recent neurological studies that show that rats revisit recent situations in a way that suggests dreaming (Wilson et al 2001). A simple extension is made to the ACS that uses the incomplete information contained in the classifier list as a basis for an abstract world model in which to interact or ’dream’. The abstract thread or dream direction is an emergent property of the selection process, this can be used to recycle around well known states and reduce real world interaction. The system is applied to two simple problems, the random walk and T−maze experiment and demonstrate that they require considerably less interactions with the real world to develop confident world models. Further models and extensions are proposed to advance the system, such as environmental directed generalisation and speculative rule creation.