There has been a proliferation of proposed mental modules in an attempt to account for different cognitive functions but so far there has been no successful account of their integration. ACT-R (Anderson & Lebiere, 1998) has evolved into a theory that consists of multiple modules but also explains how they are integrated to produce coherent cognition. The perceptual-motor modules, the goal module, and the declarative memory module are presented as examples of specialized systems in ACT-R. These modules are associated with distinct cortical regions. These modules place chunks in buffers where they can be detected by a production system that responds to patterns of information in the buffers. At any point in time a single production rule is selected to respond to the current pattern. Subsymbolic processes serve to guide the selection of rules to fire as well as the internal operations of some modules. Much of learning involves tuning of these subsymbolic processes. Empirical examples are presented that illustrate the predictions of ACT-R’s modules. In addition, two models of complex tasks are described to illustrate how these modules result in strong predictions when they are brought together. One of these models is concerned with complex patterns of behavioral data in a dynamic task and the other is concerned with fMRI data obtained in a study of symbol manipulation.

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model’s performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

The frontal cortex and basal ganglia interact via a relatively well-understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting or "releasing the brakes" on frontal motor action plans --- the basal ganglia detect appropriate contexts for performing motor actions, and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, while the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task (CPT) that requires subroutine-like selective working memory updating, and compare and contrast our model with other existing models and th...

This article describes the Adaptive Control of Thought–Rational (ACT–R) cognitive architecture (Anderson et al., 2004; Anderson & Lebiere, 1998) and its detailed application to the learning of algebraic symbol manipulation. The theory is applied to modeling the data from a study by Qin, Anderson, Silk, Stenger, & Carter (2004) in which children learn to solve linear equations and perfect their skills over a 6-day period. Functional MRI data show that: (a) a motor region tracks the output of equation solutions, (b) a prefrontal region tracks the retrieval of declarative information, (c) a parietal region tracks the transformation of mental representations of the equation, (d) an anterior cingulate region tracks the setting of goal information to control the information flow, and (e) a caudate region tracks the firing of productions in the ACT–R model. The article concludes with an architectural comparison of the competence children display in this task and the competence that monkeys have shown in tasks that require manipulations of sequences of elements.

This article presents a biologically inspired model for motor skills imitation. The model is composed of modules whose functinalities are inspired by corresponding brain regions responsible for the control of movement in primates. These modules are high-level abstractions of the spinal cord, the primary and premotor cortexes (M1 and PM), the cerebellum, and the temporal cortex. Each module is modeled at a connectionist level. Neurons in PM respond both to visual observation of movements and to corresponding motor commands produced by the cerebellum. As such, they give an abstract representation of mirror neurons. Learning of new combinations of movements is done in PM and in the cerebellum. Premotor cortexes and cerebellum are modeled by the DRAMA neural architecture which allows learning of times series and of spatio-temporal invariance in multimodal inputs. The model is implemented in a mechanical simulation of two humanoid avatars, the imitator and the imitatee. Three types of sequences learning are presented: (1) learning of repetitive patterns of arm and leg movements; (2) learning of oscillatory movements of shoulders and elbows, using video data of a human demonstration; 3) learning of precise movements of the extremities for grasp and reach

Two imaging studies were performed -- one of an algebraic transformation task studied by Anderson, Reder, and Lebiere (1996) and the other of an abstraction symbol manipulation task studied by Blessing and Anderson (1996). ACT-R models exist that carefully model the latency patterns in these tasks. These models require activity of an imaginal buffer to represent changes in the problem representation, in a retrieval buffer to hold information from declarative memory, and in a manual buffer to hold information about motor behavior. A general theory is described about how to map activity in these buffers onto the fMRI bold response. This theory claims that the BOLD response is integrated over the duration a buffer is active and can be used to predict the observed BOLD function. Activity in the imaginal buffer is shown to predict the BOLD response in a left, posterior parietal region; activity in the retrieval buffer is shown to predict the BOLD response in a left DLPFC region; and activity in the manual buffer is shown to predict activity in a motor region. Cognitive models have been increasingly successful at accounting for complex data sets on problem-solving (Anderson & Lebiere, 1998; Meyer & Kieras, 1997; Pew & Mavor, 1998). Largely, these cognitive models have focused on reaction time and accuracy and usually only final times and accuracies. These models often specify rather complex sequences of unseen processes taking place over many seconds. Even when the pattern of data they fit is correspondingly complex, one is naturally wary about a chain of inferences about unseen processes. It would be better if we could have data about these intervening processes. Basically, more converging data would be better. This paper will demonstrate the potential of functional magnetic...

The authors propose a reinforcement-learning mechanism as a model for recurrent choice and extend it to account for skill learning. The model was inspired by recent research in neurophysiological studies of the basal ganglia and provides an integrated explanation of recurrent choice behavior and skill learning. The behavior includes effects of differential probabilities, magnitudes, variabilities, and delay of reinforcement. The model can also produce the violation of independence, preference reversals, and the goal gradient of reinforcement in maze learning. An experiment was conducted to study learning of action sequences in a multistep task. The fit of the model to the data demonstrated its ability to account for complex skill learning. The advantages of incorporating the mechanism into a larger cognitive architecture are discussed.

A simplified model of the cerebellum was developed to explore its potential for adaptive, predictive control based on delayed feedback information. An abstract representation of a single Purkinje cell with multistable properties was interfaced, via a formalized premotor network, with a simulated single degree-of-freedom limb. The limb actuator was a nonlinear spring-mass system based on the nonlinear velocity dependence of the stretch reflex. By including realistic mossy fiber signals, as well as realistic conduction delays in afferent and efferent pathways, the model allowed the investigation of timing and predictive processes relevant to cerebellar involvement in the control of movement. The model regulates movement by learning to react in an anticipatory fashion to sensory feedback. Learning depends on training information generated from corrective movements and uses a temporally-asymmetric form of plasticity for the parallel fiber synapses on Purkinje cells. 1

This report is a review of the main neuro-control technologies. Two main kinds of neuro-control approaches are distinguished. One entails developing a single controller from a neural network and the other one embeds a number of controllers inside a neural network. The single neuro-control approaches are mainly system inverse: the inverse of the system dynamics is used to control the system in an open loop manner. The Multi-Layer Perceptron (MLP) is widely used for this purpose although there is no guarantee that it can succeed in learning to control the plant and that, more importantly, the unclear representation it achieves prohibits the analysis of its learned control properties. These problems and the fact that open loop control is not suitable for many systems highly restricts the usefulness of the MLP for control purposes. However, the non-linear modelling capability of the MLP could be exploited to enhance model based predictive control approaches since essentially, an accurate m...

Abstract: In this paper, we present a model for the generation of ideas within a creative thinking/brainstorming context. In the model, ideas emerge as conceptual combinations from the interaction of complex dynamics at several semantic levels: Features, concepts, categories, and previously generated ideas. This dynamics is shaped by external information on task context, constraints and goals, and is modulated by evaluative feedback from an internal critic working through reinforcement. While the model is abstract, it attempts to capture the interplay between semantic representations in the temporal, frontal and parietal cortices, working memory in the prefrontal cortex, attentional selection by the basal ganglia, and modulation from the dopaminergic reward system. We show that a context-specific itinerant search for novel but meaningful conceptual combinations (ideas) emerges naturally from the dynamics of this system. We also briefly describe a computational model for ideation in groups using a multiagent formalism. The initial focus of this model is on studying the potential benefits of cognitive diversity in agent groups, e.g., the presence of convergent and divergent thinkers, or agents with different semantic organizations. I.