Training a multilayer neural net by back-propagation is slow and requires arbitrary choices regarding the number of hidden units and layers. This paper describes an algorithm which is much faster than back-propagation and for which it is not necessary to specify the number of hidden units in advance. The relationship with other fast pattern recognition algorithms, such as algorithms based on k-d trees, is mentioned. The algorithm has been implemented and tested on articial problems such as the parity problem and on real problems arising in speech recognition. Experimental results, including training times and recognition accuracy, are given. Generally, the algorithm achieves accuracy as good as or better than nets trained using back-propagation, and the training process is much faster than back-propagation. Accuracy is comparable to that for the \nearest neighbour" algorithm, which is slower and requires more storage space. Comments Only the Abstract is given here. The full paper ap...

This paper presents new methods for training large neural networks for phoneme probability estimation. An architecture combining time-delay windows and recurrent connections is used to capture the important dynamic information of the speech signal. Because the number of connections in a fully connected recurrent network grows super-linear with the number of hidden units, schemes for sparse connection and connection pruning are explored. It is found that sparsely connected networks outperform their fully connected counterparts with an equal number of connections. The implementation of the combined architecture and training scheme is described in detail. The networks are evaluated in a hybrid HMM/ANN system for phoneme recognition on the TIMIT database, and for word recognition on the WAXHOLM database. The achieved phone error-rate, 27.8%, for the standard 39 phoneme set on the core test-set of the TIMIT database is in the range of the lowest reported. All training and simulation softwar...

Cross validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping"). The exact criterion used for cross validation based early stopping, however, is chosen in an ad-hoc fashion by most researchers or training is stopped interactively. To aid a more well-founded selection of the stopping criterion, 14 different automatic stopping criteria from 3 classes were evaluated empirically for their efficiency and effectiveness in 12 different classification and approximation tasks using multi layer perceptrons with RPROP training. The experiments show that on the average slower stopping criteria allow for small improvements in generalization (on the order of 4%), but cost about factor 4 longer training time. 1 Training for generalization When training a neural network, one is usually interested in obtaining a network with optimal generalization performance. Genera...

We describe recent extensions to our framework for the automatic generation of music-making programs. We have previously used genetic programming techniques to produce musicmaking programs that satisfy user-provided critical criteria. In this paper we describe new work on the use of connectionist techniques to automatically induce musical structure from a corpus. We show how the resulting neural networks can be used as critics that drive our genetic programming system. We argue that this framework can potentially support the induction and recapitulation of deep structural features of music. We present some initial results produced using neural and hybrid symbolic /neural critics, and we discuss directions for future work. 1 Introduction In previous work we developed a framework for the automatic generation of art-making programs on the basis of user-provided critical criteria [ Spector and Alpern, 1994 ] . Our implementation of this framework used genetic programming technology develo...

Languages vary dramatically in their structuring of space. Despite this wide variation, however, the search for universals in spatial semantics is well motivated by the fact that all linguistic spatial systems are based on human experience of space, which is in turn constrained by the nature of the human perceptual system. I present a connectionist model which contributes to the search for universals in this domain. Its design incorporates a number of structural devices motivated by neurobiological and psychophysical evidence concerning the human visual system; these provide a universal perceptual core which constrains the process of semantic acquisition. Using these structures, the model learns the perceptually grounded semantics for closed-class spatial terms from a range of languages --- providing at least a preliminary model of the human capacity for categorizing spatial events and relations. The model gives rise to two predictions concerning the manner in which one can e...

This thesis examines how artificial neural networks can benefit a large vocabulary, speaker independent, continuous speech recognition system. Currently, most speech recognition systems are based on hidden Markov models (HMMs), a statistical framework that supports both acoustic and temporal modeling. Despite their state-of-the-art performance, HMMs make a number of suboptimal modeling assumptions that limit their potential effectiveness. Neural networks avoid many of these assumptions, while they can also learn complex functions, generalize effectively, tolerate noise, and support parallelism. While neural networks can readily be applied to acoustic modeling, it is not yet clear how they can be used for temporal modeling. Therefore, we explore a class of systems called NN-HMM hybrids, in which neural networks perform acoustic modeling, and HMMs perform temporal modeling. We argue that a NN-HMM hybrid has several theoretical advantages over a pure HMM system, including better acoustic ...

A critical feature of any computer system is its interface with the user. This has led to the development of user interface technologies such as mouse, touchscreen and penbased input devices. Since handwriting is one of the most familiar communication media, pen-based interfaces combined with automatic handwriting recognition offers a very easy and natural input method. Pen-based interfaces are also essential in mobile computing because they are scalable. Recent advances in pen-based hardware and wireless communication have been influential factors in the renewed interest in on-line recognition systems. On-line handwriting recognition is fundamentally a pattern classification task; the objective is to take an input pattern, the handwritten signal collected on-line via a digitizing device, and classify it as one of a pre-specified set of words (i.e., the system's lexicon). Because exact recognition is very difficult, a lexicon is used to constrain the recognition output to a known vocab...

Software metrics provide effective methods for characterizing software. Metrics have traditionally been composed through the definition of an equation, but this approach is limited by the fact that all the interrelationships among all the parameters be fully understood. Derivation of a polynomial providing the desired characteristics is a substantial challenge. This paper explores an alternative, neural network approach to generating metrics. Experiments performed on two widely known metrics, McCabe and Halstead, indicate that the approach is sound, thus serving as the groundwork for further exploration into the analysis and design of software metrics.

This paper demonstrates how the backpropagation algorithm #BP# and its variants can

this article, we describe and develop methodologies for mod- eling and transferring human control strategy (HCS). This research has potential application in a variety of areas such as the Intelligent Vehicle Highway System (IVHS), human-machine interfacing, real-time training, space telerobotics, and agile manufacturing. We specifically address the following issues: (1) how to efficiently model human control strategy through learning cascade neural networks, (2) how to select state inputs in order to generate reliable models, (3) how to validate the computed models through an independent, Hidden Markov Model-based procedure, and (4) how to effectively transfer human control strategy. We have implemented this approach experimentally in the real-time control of a human driving simulator, and are working to transfer these methodologies for the control of an autonomous vehicle and a mobile robot. In providing a framework for abstracting computational models of human skill, we expect to facilitate analysis of human control, the development of humanlike intelligent machines, improved human-robot coordination, and the transfer of skill from one human to another