Results 1  10
of
24
Parallel Networks that Learn to Pronounce English Text
 COMPLEX SYSTEMS
, 1987
"... This paper describes NETtalk, a class of massivelyparallel network systems that learn to convert English text to speech. The memory representations for pronunciations are learned by practice and are shared among many processing units. The performance of NETtalk has some similarities with observed h ..."
Abstract

Cited by 456 (5 self)
 Add to MetaCart
This paper describes NETtalk, a class of massivelyparallel network systems that learn to convert English text to speech. The memory representations for pronunciations are learned by practice and are shared among many processing units. The performance of NETtalk has some similarities with observed human performance. (i) The learning follows a power law. (;i) The more words the network learns, the better it is at generalizing and correctly pronouncing new words, (iii) The performance of the network degrades very slowly as connections in the network are damaged: no single link or processing unit is essential. (iv) Relearning after damage is much faster than learning during the original training. (v) Distributed or spaced practice is more effective for longterm retention than massed practice. Network models can be constructed that have the same performance and learning characteristics on a particular task, but differ completely at the levels of synaptic strengths and singleunit responses. However, hierarchical clustering techniques applied to NETtalk reveal that these different networks have similar internal representations of lettertosound correspondences within groups of processing units. This suggests that invariant internal representations may be found in assemblies of neurons intermediate in size between highly localized and completely distributed representations.
A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features
 Machine Learning
, 1993
"... In the past, nearest neighbor algorithms for learning from examples have worked best in domains in which all features had numeric values. In such domains, the examples can be treated as points and distance metrics can use standard definitions. In symbolic domains, a more sophisticated treatment of t ..."
Abstract

Cited by 266 (3 self)
 Add to MetaCart
In the past, nearest neighbor algorithms for learning from examples have worked best in domains in which all features had numeric values. In such domains, the examples can be treated as points and distance metrics can use standard definitions. In symbolic domains, a more sophisticated treatment of the feature space is required. We introduce a nearest neighbor algorithm for learning in domains with symbolic features. Our algorithm calculates distance tables that allow it to produce realvalued distances between instances, and attaches weights to the instances to further modify the structure of feature space. We show that this technique produces excellent classification accuracy on three problems that have been studied by machine learning researchers: predicting protein secondary structure, identifying DNA promoter sequences, and pronouncing English text. Direct experimental comparisons with the other learning algorithms show that our nearest neighbor algorithm is comparable or superior ...
Rethinking Eliminative Connectionism
, 1998
"... Humans routinely generalize universal relationships to unfamiliar instances. If we are told ‘‘if glork then frum,’ ’ and ‘‘glork,’ ’ we can infer ‘‘frum’’; any name that serves as the subject of a sentence can appear as the object of a sentence. These universals are pervasive in language and reasoni ..."
Abstract

Cited by 65 (4 self)
 Add to MetaCart
Humans routinely generalize universal relationships to unfamiliar instances. If we are told ‘‘if glork then frum,’ ’ and ‘‘glork,’ ’ we can infer ‘‘frum’’; any name that serves as the subject of a sentence can appear as the object of a sentence. These universals are pervasive in language and reasoning. One account of how they are generalized holds that humans possess mechanisms that manipulate symbols and variables; an alternative account holds that symbolmanipulation can be eliminated from scientific theories in favor of descriptions couched in terms of networks of interconnected nodes. Can these ‘‘eliminative’ ’ connectionist models offer a genuine alternative? This article shows that eliminative connectionist models cannot account for how we extend universals to arbitrary items. The argument runs as follows. First, if these models, as currently conceived, were to extend universals to arbitrary instances, they would have to generalize outside the space of training examples. Next, it is shown that the class of eliminative connectionist models that is currently popular cannot learn to extend universals outside the training space. This limitation might be avoided through the use of an architecture that implements symbol manipulation.
The Dynamics of Nonlinear Relaxation Labeling Processes
, 1997
"... We present some new results which definitively explain the behavior of the classical, heuristic nonlinear relaxation labeling algorithm of Rosenfeld, Hummel, and Zucker in terms of the HummelZucker consistency theory and dynamical systems theory. In particular, it is shown that, when a certain symm ..."
Abstract

Cited by 31 (10 self)
 Add to MetaCart
We present some new results which definitively explain the behavior of the classical, heuristic nonlinear relaxation labeling algorithm of Rosenfeld, Hummel, and Zucker in terms of the HummelZucker consistency theory and dynamical systems theory. In particular, it is shown that, when a certain symmetry condition is met, the algorithm possesses a Liapunov function which turns out to be (the negative of) a wellknown consistency measure. This follows almost immediately from a powerful result of Baum and Eagon developed in the context of Markov chain theory. Moreover, it is seen that most of the essential dynamical properties of the algorithm are retained when the symmetry restriction is relaxed. These properties are also shown to naturally generalize to higherorder relaxation schemes. Some applications and implications of the presented results are finally outlined.
Perseverative and Semantic Influences on Visual Object Naming Errors in Optic Aphasia: A Connectionist Account
 JOURNAL OF COGNITIVE NEUROSCIENCE
, 1993
"... Although perseverationthe inappropriate repetition of previous responsesis quite common among patients with neurological damage, relatively few detailed computational accounts of its various forms have been put forth. A particularly welldocumented variety involves the pattern of errors made ..."
Abstract

Cited by 28 (8 self)
 Add to MetaCart
Although perseverationthe inappropriate repetition of previous responsesis quite common among patients with neurological damage, relatively few detailed computational accounts of its various forms have been put forth. A particularly welldocumented variety involves the pattern of errors made by "optic aphasic" patients, who have a selective deficit in naming visuallypresented objects. Based on our previous work in modeling impaired reading for meaning in deep dyslexia, we develop a connectionist simulation of visual object naming. The major extension in the present work is the incorporation of shortterm correlational weights that bias the network towards reproducing patterns of activity that have occurred on recently preceding trials. Under damage, the network replicates the complex semantic and perseverative effects found in the optic aphasic error pattern. Further analysis reveals that the perseverative effects are strongest when the lesions are near or within semanti...
The How and Why of What Went Where in Apparent Motion: Modeling Solutions to the Motion Correspondence Problem
 PSYCHOLOGICAL REVIEW
, 1991
"... A model that is capable of maintaining the identities of individuated elements as they move is described. It solves a particular problem of underdetermination, the motion correspondence problem, by simultaneously applying 3 constraints: the nearest neighbor principle, the relative velocity princip ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
A model that is capable of maintaining the identities of individuated elements as they move is described. It solves a particular problem of underdetermination, the motion correspondence problem, by simultaneously applying 3 constraints: the nearest neighbor principle, the relative velocity principle, and the element integrity principle. The model generates the same correspondence solutions as does the human visual system for a variety of displays, and many of its properties are consistent with what is known about the physiological mechanisms underlying human motion perception. The model can also be viewed as a proposal of how the identities of attentional tags are maintained by visual cognition, and thus it can be differentiated from a system that serves merely to detect movement.
On the complexity of learning from counterexamples and membership queries
 In 31st Annual Symposium on Foundations of Computer Science
, 1990
"... We show that for any concept class C the number of equivalence and membership queries that are needed to learn C is bounded from below by R(VCdimension(C)). Furthermore we show that the required number of equivalence and membership queries is also bounded from below by R(LC ARB(C) / log(1 + LC ..."
Abstract

Cited by 17 (0 self)
 Add to MetaCart
We show that for any concept class C the number of equivalence and membership queries that are needed to learn C is bounded from below by R(VCdimension(C)). Furthermore we show that the required number of equivalence and membership queries is also bounded from below by R(LC ARB(C) / log(1 + LC ARB(C))), where LC ARB(C) is the required number of steps in a different mo+l where no membership queries but equivalence queries with arbitrary subsets of the domain are permitted. These two relationships are the only relationships between the learning complexities of the common online learning models and the related combinatorial parameters that have remained open (see section 3 of [MTl]). As an application of the first lower bound we determine the number of equivalence and membership queries that are needed to learn monomials of IC out of n variables. In the last section we examine learning algorithms for threshold gates that are based on equivalence queries. We show that a threshold gate can not only learn concepts but also nondecreasing functions in polynomially many steps. On the other hand we show that all distributed learning algorithms for threshold gates that are of a similar type as the Arule or the WINNOWalgorithm are inherently slow.
The Connectivity of the Brain: MultiLevel Quantitative Analysis
 Biological Cybernetics
, 1995
"... We develop a mathematical formalism for calculating connectivity volumes generated by specific topologies with various physical packing strategies. We consider four topologies (full, random, nearest neighbor, and modular connectivity) and three physical models: (i) interior packing, where neurons a ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
We develop a mathematical formalism for calculating connectivity volumes generated by specific topologies with various physical packing strategies. We consider four topologies (full, random, nearest neighbor, and modular connectivity) and three physical models: (i) interior packing, where neurons and connection fibers are intermixed, (ii) sheeted packing where neurons are located on a sheet with fibers running underneath, and (iii) exterior packing where the neurons are located at the surfaces of a cube or sphere with fibers taking up the internal volume. By extensive crossreferencing of available human neuroanatomical data we produce a consistent set of parameters for the whole brain, the cerebral cortex, and the cerebellar cortex. By comparing these inferred values with those predicted by the expressions, we draw the following general conclusions for the human brain, cortex, cerebellum: (i) Interior packing is less efficient than exterior packing (in a sphere). (ii) Fully and rando...
A Feedback Model of Visual Attention
 JOURNAL OF COGNITIVE NEUROSCIENCE
, 2004
"... Feedback connections are a prominent feature of cortical anatomy and are likely to have significant functional role in neural information processing. We present a neural network model of cortical feedback that successfully simulates neurophysiological data associated with attention. In this domain o ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
Feedback connections are a prominent feature of cortical anatomy and are likely to have significant functional role in neural information processing. We present a neural network model of cortical feedback that successfully simulates neurophysiological data associated with attention. In this domain our model can be considered a more detailed, and biologically plausible, implementation of the biased competition model of attention. However, our model is more general as it can also explain a variety of other topdown processes in vision, such as figure/ground segmentation and contextual cueing. This model thus suggests that a common mechanism, involving cortical feedback pathways, is responsible for a range of phenomena and provides a unified account of currently disparate areas of research.