Results 1  10
of
14
Language as a Dynamical System
 In
, 1995
"... Introduction Despite considerable diversity among theories about how humans process language, there are a number of fundamental assumptions which are shared by most such theories. This consensus extends to the very basic question about what counts as a cognitive process. So although many cognitive s ..."
Abstract

Cited by 79 (2 self)
 Add to MetaCart
(Show Context)
Introduction Despite considerable diversity among theories about how humans process language, there are a number of fundamental assumptions which are shared by most such theories. This consensus extends to the very basic question about what counts as a cognitive process. So although many cognitive scientists are fond of referring to the brain as a `mental organ' (e.g., Chomsky, 1975)implying a similarity to other organs such as the liver or kidneysit is also assumed that the brain is an organ with special properties which set it apart. Brains `carry out computation' (it is argued)
Bounds for the Computational Power and Learning Complexity of Analog Neural Nets
 Proc. of the 25th ACM Symp. Theory of Computing
, 1993
"... . It is shown that high order feedforward neural nets of constant depth with piecewise polynomial activation functions and arbitrary real weights can be simulated for boolean inputs and outputs by neural nets of a somewhat larger size and depth with heaviside gates and weights from f\Gamma1; 0; 1g. ..."
Abstract

Cited by 56 (12 self)
 Add to MetaCart
(Show Context)
. It is shown that high order feedforward neural nets of constant depth with piecewise polynomial activation functions and arbitrary real weights can be simulated for boolean inputs and outputs by neural nets of a somewhat larger size and depth with heaviside gates and weights from f\Gamma1; 0; 1g. This provides the first known upper bound for the computational power of the former type of neural nets. It is also shown that in the case of first order nets with piecewise linear activation functions one can replace arbitrary real weights by rational numbers with polynomially many bits, without changing the boolean function that is computed by the neural net. In order to prove these results we introduce two new methods for reducing nonlinear problems about weights in multilayer neural nets to linear problems for a transformed set of parameters. These transformed parameters can be interpreted as weights in a somewhat larger neural net. As another application of our new proof technique we s...
A Weak Version of the Blum, Shub & Smale Model
, 1994
"... We propose a weak version of the BlumShubSmale model of computation over the real numbers. In this weak model only a "moderate" usage of multiplications and divisions is allowed. The class of boolean languages recognizable in polynomial time is shown to be the complexity class P/poly ..."
Abstract

Cited by 30 (6 self)
 Add to MetaCart
We propose a weak version of the BlumShubSmale model of computation over the real numbers. In this weak model only a "moderate" usage of multiplications and divisions is allowed. The class of boolean languages recognizable in polynomial time is shown to be the complexity class P/poly. The main tool is a result on the existence of small rational points in semialgebraic sets which is of independent interest. As an application, we generalize recent results of Siegelmann & Sontag on recurrent neural networks, and of Maass on feedforward nets. A preliminary version of this paper was presented at the 1993 IEEE Symposium on Foundations of Computer Science. Additional results include: \Pi an efficient simulation of orderfree real Turing machines by probabilistic Turing machines in the full BlumShubSmale model; \Pi an efficient simulation of arithmetic circuits over the integers by boolean circuits; \Pi the strict inclusion of the real polynomial hierarchy in weak exponentia...
On the power of real Turing machines over binary inputs
 SIAM Journal on Computing
, 1997
"... this paper is to prove that BP(PAR IR ) = PSPACE/poly where PAR IR is the class of sets computed in parallel polynomial time by (ordinary) real Turing machines. As a consequence we obtain the existence of binary sets that do not belong to the Boolean part of PAR IR (an extension of the result in [20 ..."
Abstract

Cited by 25 (4 self)
 Add to MetaCart
(Show Context)
this paper is to prove that BP(PAR IR ) = PSPACE/poly where PAR IR is the class of sets computed in parallel polynomial time by (ordinary) real Turing machines. As a consequence we obtain the existence of binary sets that do not belong to the Boolean part of PAR IR (an extension of the result in [20] since PH IR ` PAR IR ) and a separation of complexity classes in the real setting.
On the Complexity of Training Neural Networks with Continuous Activation Functions
, 1993
"... We deal with computational issues of loading a fixedarchitecture neural network with a set of positive and negative examples. This is the first result on the hardness of loading networks which do not consist of the binarythreshold neurons, but rather utilize a particular continuous activation func ..."
Abstract

Cited by 22 (2 self)
 Add to MetaCart
We deal with computational issues of loading a fixedarchitecture neural network with a set of positive and negative examples. This is the first result on the hardness of loading networks which do not consist of the binarythreshold neurons, but rather utilize a particular continuous activation function, commonly used in the neural network literature. We observe that the loading problem is polynomialtime if the input dimension is constant. Otherwise, however, any possible learning algorithm based on particular fixed architectures faces severe computational barriers. Similar theorems have already been proved by Megiddo and by Blum and Rivest, to the case of binarythreshold networks only. Our theoretical results lend further justification to the use of incremental (architecturechanging) techniques for training networks rather than fixed architectures. Furthermore, they imply hardness of learnability in the probablyapproximatelycorrect sense as well.
Hill Climbing in Recurrent Neural Networks for Learning the a^n b^n c^n Language
"... A simple recurrent neural network is trained on a onestep look ahead prediction task for symbol sequences of the contextsensitive a n b n c n language. Using an evolutionary hill climbing strategy for incremental learning the network learns to predict sequences of strings up to depth n = 12. ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
A simple recurrent neural network is trained on a onestep look ahead prediction task for symbol sequences of the contextsensitive a n b n c n language. Using an evolutionary hill climbing strategy for incremental learning the network learns to predict sequences of strings up to depth n = 12. Experiments and the algorithms used are described. The activation of the hidden units of the trained network is displayed in a 3D graph and analysed.
On the Relations Between Dynamical Systems and Boolean Circuits
, 1993
"... We study the computational capabilities of dynamical systems defined by iterated functions on [0, 1]^n. The computations are performed with infinite precision on arbitrary real numbers, like in the model of analog computation recently proposed by Hava Siegelmann and Eduardo Sontag. We concentrate ma ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We study the computational capabilities of dynamical systems defined by iterated functions on [0, 1]^n. The computations are performed with infinite precision on arbitrary real numbers, like in the model of analog computation recently proposed by Hava Siegelmann and Eduardo Sontag. We concentrate mainly on the lowdimensional case and on the relations with the BlumShubSmale model of computation over the real numbers.
On the Intractability of Loading Neural Networks
 Theoretical Advances in Neural Computation and Learning
, 1994
"... Introduction Neural networks have been proposed as a tool for machine learning. In this role, a network is trained to recognize complex associations between inputs and outputs that were presented during a supervised training cycle. These associations are incorporated into the weights of the network ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
Introduction Neural networks have been proposed as a tool for machine learning. In this role, a network is trained to recognize complex associations between inputs and outputs that were presented during a supervised training cycle. These associations are incorporated into the weights of the network, which encode a distributed representation of the information that was contained in the patterns. Once trained, the network will compute an input/output mapping which, if the training data was representative enough, will closely match the unknown rule which produced the original data. Massive parallelism of computation, as well as noise and fault tolerance, are often offered as justifications for the use of neural nets as learning paradigms. By "neural network" we always mean, in this chapter, feedforward ones of the type routinely employed in artificial neural nets applications. That is, a net consists of a number of processors ("nodes" or "neurons") each of which computes a functi
Computing on Analog Neural Nets with Arbitrary Real Weights
 in : Theoretical Advances in Neural Computation and Learning
, 1994
"... ..."
(Show Context)