Two connectionist models of attention in associative learning, previously used to model human category learning, are shown to have special cases that are essentially equivalent to N. J. Mackintosh's (1975, Psychological Review, 82, 276 298) classic model of attention in animal learning. The models unify formulas for associative weight change with formulas for attentional change, under a common goal of error reduction. Error-driven attentional shifting accelerates learning of new associations but also protects previously learned associations from retroactive interference. The models are fit to data from a recent experiment in human associative learning (J. K. Kruschke 6 N. J. Blair, 2000, Psychonomic Bulletin 6 Review, 7, 636 645), which shows that blocking of learning involves learned inattention. The approach also provides a novel and unifying theory of latent inhibition (the preexposure effect) in terms of blocking. The discussion summarizes how the approach accounts for a variety of other ``irrational' ' phenomena in associative learning, including base rate effects, perseveration of attention through relevance

In everyday tasks, selecting actions in the proper sequence requires a continuously updated representation of temporal context. Many existing models address this problem by positing a hierarchy of processing units, mirroring the roughly hierarchical structure of naturalistic tasks themselves. Such an approach has led to a number of difficulties, including a reliance on overly rigid sequencing mechanisms, an inability to account for context sensitivity in behavior, and a failure to address learning. We consider here an alternative framework, according to which the representation of temporal context is facilitated by recurrent connections within a network mapping from environmental inputs to actions. Applying this approach to a specific, and in many ways prototypical, everyday task (coffee-making), we examine its ability to account for several central characteristics of normal and impaired human performance. The model we consider learns to deal flexibly with a complex set of sequencing constraints, encoding contextual information at multiple time-scales within a single, distributed internal representation. Mildly degrading this context representation leads

the properties of distributed network models, and support this account by demonstrating that an implemented simulation closely approximates the empirical findings despite the absence of expectancy-based processes and postlexical semantic matching. The results suggest that distributed network models can provide a viable single-mechanism account of lexical processing. Introduction It is well-established that people are faster and more accurate to read a word (e.g., BUTTER) when it is preceded by a related word (e.g., BREAD) compared with when it is preceded by an unrelated word (e.g., DOCTOR; The research was supported by an NIMH FIRST award (MH55628) to the first author and by NIMH Training Grant 5T32MH19102 and NICHD Grant 80258. The computational simulation was run using customized software written within the Xerion simulator (version 3.1) developed by Drew van Camp, Tony Plate, and Geoff Hinton at the Univers

this article, we have presented evidence that a computational model that instantiates approximate, local learning with graded transfer provides a good account of how subjects learn on-line from outcome feedback in the SPF, a simple dynamic task. We base this conclusion on the model's ability to predict subjects' performance during training and on two subsequent tests of their ability to generalize, the control questions and the transfer task. We now explore the limitations of our efforts and discuss two alternative approaches to understanding human performance before concluding on our own approach 's merits

Neural networks have emerged as a field

Logistic regression (LR), discriminant analysis (DA), and neural networks (NN) were used to predict ordered and disordered regions in proteins. Training data were from a set of non-redundant X-ray crystal structures, with the data being partitioned into N-terminal, C-terminal and internal (I) regions. The DA and LR methods gave almost identical 5-cross validation accuracies that averaged to the following values: 75.9 ± 3.1 % (N-regions), 70.7 ± 1.5 % (I-regions), and 74.6 ± 4.4 % (C-regions). NN predictions gave slightly higher scores: 78.8 ± 1.2 % (N-regions), 72.5 ± 1.2 % (I-regions), and 75.3 ± 3.3 % (C-regions). Predictions improved with length of the disordered regions. Averaged over the three methods, values ranged from 52 % to 78 % for length = 9-14 to ≥ 21, respectively, for I-regions, from 72 % to 81 % for length = 5 to 12-15, respectively, for N-regions, and from 70 % to 80 % for length = 5 to 12-15, respectively, for C-regions. These data support the hypothesis that disorder is encoded by the amino acid sequence. 1

. The calculation of second derivatives is required by recent training and analyses techniques of connectionist networks, such as the elimination of superfluous weights, and the estimation of confidence intervals both for weights and network outputs. We here review and develop exact and approximate algorithms for calculating second derivatives. For networks with jwj weights, simply writing the full matrix of second derivatives requires O(jwj 2 ) operations. For networks of radial basis units or sigmoid units, exact calculation of the necessary intermediate terms requires of the order of 2h + 2 backward/forward-propagation passes where h is the number of hidden units in the network. We also review and compare three approximations (ignoring some components of the second derivative, numerical differentiation, and scoring). Our algorithms apply to arbitrary activation functions, networks, and error functions (for instance, with connections that skip layers, or radial basis functions, or ...

Abstract not found

This paper describes a learning news agent HyNeT which uses hybrid neural network techniques for classifying news titles as they appear on an internet newswire. Recurrent plausibility networks with local memory are developed and examined for learning robust text routing. HyNeT is described for the first time in this paper. We show that a careful hybrid integration of techniques from neural network architectures, learning and information retrieval can reach consistent recall and precision rates of more than 92% on an 82 000 word corpus; this is demonstrated for 10 000 unknown news titles from the Reuters newswire. This new synthesis of neural networks, learning and information retrieval techniques allows us to scale up to a real-world task and demonstrates a lot of potential for hybrid plausibility networks for semantic text routing agents on the internet. Introduction In the last decade, a lot of work on neural networks in artificial intelligence has focused on fundam...

Many scientific and industrial problems can be better understood by learning from samples of the task at hand. For this reason, the machine learning and statistics communities devote considerable research effort on generating inductive-learning algorithms that try to learn the true "concept" of a task from a set of its examples. Often times, however, one has additional resources readily available, but largely unused, that can improve the concept that these learning algorithms generate. These resources include available computer cycles, as well as prior knowledge describing what is currently known about the domain. Effective utilization of available computer time is important since for most domains an expert is willing to wait for weeks, or even months, if a learning system can produce an improved concept. Using prior knowledge is important since it can contain information not present in the current set of training examples. In this thesis, I present three "anytime" approaches to connec...