Introduction Backpropagation is probably the most widely applied neural network learning algorithm. Backprop's popularity is related to its ability to deal with complex multi-dimensional mappings. In the words of Werbos [56] the algorithm goes \beyond regression". Backprop 's theory is related to many disciplines and has been developed by several dierent research groups. As pointed out by le Cun [38], to some extent, the basic elements of the theory can be traced back to the famous book of Bryson and Ho[9]. A more explicit statement of the algorithm has been proposed by Werbos [56], Parker [43], le Cun [36], and members of the PDP group [44]. Although many researchers have contributed in dierent ways in the development and proposition of dierent aspects of Backprop, there is no question that Rumelhart and the PDP group have the credit for the current high diusion of the algorithm. As Widrow points out in [57], what is actually new with Backprop is the adoption of \squashing

. Binary classification is typically achieved by supervised learning methods. Nevertheless, it is also possible using unsupervised schemes. This paper describes a connectionist unsupervised approach to binary classification and compares its performance to that of its supervised counterpart. The approach consists of training an autoassociator to reconstruct the positive class of a domain at the output layer. After training, the autoassociator is used for classification, relying on the idea that if the network generalizes to a novel instance, then this instance must be positive, but that if generalization fails, then the instance must be negative. When tested on three real-world domains, the autoassociator proved more accurate at classification than its supervised counterpart, MLP, on two of these domains and as accurate on the third. The paper seeks to generalize these results and concludes that, in addition to learning a concept in the absence of negative examples, 1) autoassociation i...

The effectiveness of connectionist models in emulating intelligent behaviour and solving significant practical problems is strictly related to the capability of the learning algorithms to find optimal or near-optimal solutions and to generalize to new examples. This paper reviews some theoretical contributions to optimal learning in the attempt to provide a unified view and give the state of the art in the field. The focus of the review is on the problem of local minima in the cost function that is likely to affect more or less any learning algorithm. Starting from this analysis, we briefly review proposals for discovering optimal solutions and suggest conditions for designing architectures tailored to a given task. 1 Introduction In the last few years impressive efforts have been made in using connectionist models either for modelling human behaviour and for solving practical problems. In the field of cognitive science and psychology, we have been witnessing a debate on the actual ro...

Neural Network models have received increased attention in the recent years. Aimed at achieving human-like performance in tasks of the cognitive sciences domain, these models are composed of a highly interconnected mesh of nonlinear computing elements, whose structure is drawn from our current knowledge of biological neural systems. Several Neural Network Learning Algorithms have been developed in the past years. In these algorithms, a set of rules defines the evolution process undertaken by the synaptic connections of the networks, thus allowing them to learn how to perform specified tasks. In this article, several such algorithms are surveyed. They range from simple associative learning paradigms to more complex reinforcement learning systems. A detailed description of each algorithm is presented, and a discussion of their capabilities and limitations is included. 1 Introduction The history of artificial neural networks starts with the work by McCulloch and Pitts [64] in which neur...

Recent evidence (Maye, Werker & Gerken, 2002) suggests that statistical learning may be an important mechanism for the acquisition of phonetic categories in the infant’s native language. We examined the sufficiency of this hypothesis and its implications for development by implementing a statistical learning mechanism in a computational model based on a mixture of Gaussians (MOG) architecture. Statistical learning alone was found to be insufficient for phonetic category learning – an additional competition mechanism was required in order for the categories in the input to be successfully learnt. When competition was added to the MOG architecture, this class of models successfully accounted for developmental enhancement and loss of sensitivity to phonetic contrasts. Moreover, the MOG with competition model was used to explore a potentially important distributional property of early speech categories – sparseness – in which portions of the space between phonetic categories are unmapped. Sparseness was found in all successful models and quickly emerged during development even when the initial parameters favoured continuous representations with no gaps. The implications of these models for phonetic category learning in infants are discussed.

KEYWORDS: speaker verification; autoassociative neural network; distribution estimation; matching technique; dimensionality reduction.

We provide a conceptual framework for understanding similarities and differences among various schemes of compositional representation, emphasizing problems that arise in modelling aspects of human language. We propose six abstract dimensions that suggest a space of possible compositional schemes. Temporality turns out to play a key role in defining several of these dimensions. From studying how schemes fall into this space, it is apparent that there is no single crucial difference between AI and connectionist approaches to representation. Large regions of the space of compositional schemes remain unexplored, such as the entire class of active, dynamic models that do composition in time. These models offer the possibility of parsing real-time input into useful segments, and thus potentially into linguistic units like words and phrases. Introduction What is the relationship between the kinds of symbolic representations deployed in "classical" cognitive models and repr...

In this paper, we propose a new general framework for learning and recognizing spatiotemporal events (or patterns) from intensity image sequences. This scheme is general in that it does not impose any motion model on the input. A multiclass, multivariate discriminant analysis technique has been used to automatically select the most discriminating features (MDF) which is shown to be better suited for classification due to its capability to automatically discount factors that are irrelevant to classification. The space partition tree introduced here achieves a logarithmic time complexity for a database of n items. A general interpolation scheme is employed for inference and generalization in the MDF space based on a small number of training samples. The system is tested to recognize 28 different hand signs. The experimental results show that the learned system can achieve a 98% recognition rate for test sequences that have not been used in the training phase. 1 1 Introduction Temporal...

Is knowledge acquired implicitly abstract or based on memory for exemplars? This question is at the heart of a current, but long-standing, controversy in the field of implicit learning (see Reber, 1989, for a review). For some authors, implicit knowledge is best characterized as rulelike. For others, however, knowledge acquired implicitly is little more than knowledge about memorized exemplars, or at best, knowledge about elementary features of the material, such as the frequency of particular events. In this paper, I argue that the debate may be ill-posed, and that the two positions are not necessarily incompatible. Using simulation studies, I show that abstract knowledge about the stimulus material may emerge through the operation of elementary, associationist learning mechanisms of the kind that operate in connectionist networks. I focus on a sequence learning task first proposed by Kushner, Cleeremans & Reber (1991), during which subjects are exposed to random fixed-length sequence...

Psychophysical findings accumulated over the past several decades indicate that perceptual tasks such as similarity judgment tend to be performed on a low-dimensional representation of the sensorydata. Low dimensionality is especially important for learning, as the number of examples required for attaining a given level of performance grows exponentially with the dimensionality of the underlying representation space. Because of this curse of dimensionality, in shape categorization the high initial dimensionality of the sensory data must be reduced by a nontrivial computational process, which, ideally, should capture the intrinsic low-dimensional nature of families of visual shapes. We showhow to make a connectionist systemuse class labels to learn a representation that fulfills this requirement, thereby facilitating shape categorization. Our results indicate that low-dimensional representations are best extracted in a learning task that combines discrimination and generalization constr...