The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model’s performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

The frontal cortex and basal ganglia interact via a relatively well-understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting or "releasing the brakes" on frontal motor action plans --- the basal ganglia detect appropriate contexts for performing motor actions, and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, while the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task (CPT) that requires subroutine-like selective working memory updating, and compare and contrast our model with other existing models and th...

We propose a systems-level computational model of the basal ganglia based closely on known anatomy and physiology. First, we assume that the thalamic targets, which relay ascending information to cortical action and planning areas, are tonically inhibited by the basal ganglia. Second, we assume that the output stage of the basal ganglia, the internal segment of the globus pallidus (GPi), selects a single action from several competing actions via lateral interactions. Third, we propose that a form of local working memory exists in the form of reciprocal connections between the external globus pallidus (GPe) and the subthalamic nucleus (STN). As a test of the model, the system was trained to learn a sequence of states that required the context of previous actions. The striatum, which was assumed to represent a conjunction of cortical states, directly selected the action in the GP during training. The STN-to-GP connection strengths were modi�ed by an associative learning

n Recent work suggests that �uctuations in dopamine delivery at target structures represent an evaluation of future events that can be used to direct learning and decision-making. To examine the behavioral consequences of this interpretation, we gave simple decision-making tasks to 66 human subjects and to a network based on a predictive model of mesencephalic dopamine systems. The human subjects displayed behavior similar to the network behavior in terms of choice allocation and the character of deliberation times. The agreement between human and model performances suggests a direct relationship between biases in human decision strategies and �uctuating dopamine delivery. We also show that the model offers a new interpretation of de�cits that result when dopamine levels are increased or decreased through disease or pharmacological interventions. The bottom-up approach presented here also suggests that a variety of behavioral strategies may result from the expression of relatively simple neural mechanisms in different behavioral contexts. n

It is argued that a novel control architecture, the Static and Dynamic State (SDS) feedback scheme, which utilizes speed-field tracking, exhibits global stability, and allows on-line tuning by any adaptation mechanism without cancelling stability if certain structural conditions are met, can be viewed as a model of basal ganglia-thalamocortical loops since (1) the SDS scheme predicts the neuronal groups that fit neuronal classification in the supplementary motor area, the motor cortex and the putamen, (2) the structural stability conditions require parallel channels, a feature that these loops provide, and (3) the SDS scheme predicts two major disorders that can be identified as Parkinson's and Huntington 's diseases. Simulations suggests that the basal ganglia work outside the realm of the stability condition allowed by the robustness of the scheme and required for increased computation speeds.

this article, we explore the question of how temporal information is represented, both from psychological and neurological perspectives. Moreover, we consider how such representations are integrated and utilized by a more general system required for the control of coordinated action

The final copy of this thesis has been examined by the signatories and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline. Learning Working Memory Tasks by Reward Prediction iii

If robots are ever to migrate away from factory floors, light-weight concepts are required. However, the novel actuator structures used for such robots results in increasingly difficult dynamics. Control theory cannot cope without learning anymore, but even parameterized models are too inexact to be used. We present the problematics in current day robotics. 1.1 Introduction Recent successes in robotics have increased the field of application and acceptation of robots. Nevertheless, industrial robotics still has to go a long way. The applicability of classical robots remains limited to factory floors. Research lab robotics is increasingly moving towards novel actuator schemes with motors which have a high force-to-weight ratio, are rather small (about 1" \Theta 1" \Theta 2"), and are therefore useful in the construction of light-weight robot arms. Due to the elasticity of the materials used and the compliance (which is also changing due to wear-and-tear), actuators consisting of agonis...

Machines (HAMs) state theorems for basic version of offline and online learning discuss generalization learning HAM structure]] [[From 1997 LIS proposal: By combining work in control theory, model learning, and AI reinforcement learning, we are developing a mathematical model for the learning of complex control strategies that captures all of these features. The key aspect is learning hierarchical strategies that are flexible and context-sensitive. Recent work by Parr and Russell (ML-97) has shown 7 NO FIGURE YET NO FIGURE YET (a) (b) Figure 2: (a) Design of the coordinating controller. (b) Flow chart showing the relationships between the processes of action selection, simulation, and learning, for the instance of diving. that this can be achieved within the formal framework of reinforcement learning and Markov decision processes, via the specification of hierarchies of controllers with undetermined choice points (such as which motions to execute in a dive) that define an abs...

Abstract not found