Results 1  10
of
20
Stable Encoding of FiniteState Machines in DiscreteTime Recurrent Neural Nets with Sigmoid Units
, 1998
"... In recent years, there has been a lot of interest in the use of discretetime recurrent neural nets (DTRNN) to learn finitestate tasks, with interesting results regarding the induction of simple finitestate machines from inputoutput strings. Parallel work has studied the computational power of DT ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
In recent years, there has been a lot of interest in the use of discretetime recurrent neural nets (DTRNN) to learn finitestate tasks, with interesting results regarding the induction of simple finitestate machines from inputoutput strings. Parallel work has studied the computational power of DTRNN in connection with finitestate computation. This paper describes a simple strategy to devise stable encodings of finitestate machines in computationally capable discretetime recurrent neural architectures with sigmoid units, and gives a detailed presentation on how this strategy may be applied to encode a general class of finitestate machines in a variety of commonlyused first and secondorder recurrent neural networks. Unlike previous work that either imposed some restrictions to state values, or used a detailed analysis based on fixedpoint attractors, the present approach applies to any positive, bounded, strictly growing, continuous activation function, and uses simple bounding criteri...
The Computational Power of Discrete Hopfield Nets with Hidden Units
 Neural Computation
, 1996
"... We prove that polynomial size discrete Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial spacebounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks wi ..."
Abstract

Cited by 11 (6 self)
 Add to MetaCart
We prove that polynomial size discrete Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial spacebounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks with polynomially bounded interconnection weights compute exactly the class of functions P/poly, i.e., the class computed by polynomial timebounded nonuniform Turing machines.
An Overview Of The Computational Power Of Recurrent Neural Networks
 Proceedings of the 9th Finnish AI Conference STeP 2000{Millennium of AI, Espoo, Finland (Vol. 3: "AI of Tomorrow": Symposium on Theory, Finnish AI Society
, 2000
"... INTRODUCTION The two main streams of neural networks research consider neural networks either as a powerful family of nonlinear statistical models, to be used in for example pattern recognition applications [6], or as formal models to help develop a computational understanding of the brain [10]. His ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
INTRODUCTION The two main streams of neural networks research consider neural networks either as a powerful family of nonlinear statistical models, to be used in for example pattern recognition applications [6], or as formal models to help develop a computational understanding of the brain [10]. Historically, the brain theory interest was primary [32], but with the advances in computer technology, the application potential of the statistical modeling techniques has shifted the balance. 1 The study of neural networks as general computational devices does not strictly follow this division of interests: rather, it provides a general framework outlining the limitations and possibilities aecting both research domains. The prime historic example here is obviously Minsky's and Papert's 1969 study of the computational limitations of singlelayer perceptrons [34], which was a major inuence in turning away interest from neural network learning to symbolic AI techniques for more
A ContinuousTime Hopfield Net Simulation of Discrete Neural Networks
, 2000
"... We investigate the computational power of continuoustime symmetric Hopfield nets. As is well known, such networks have very constrained, Liapunovfunction controlled dynamics. Nevertheless, we show that they are universal and efficient computational devices, in the sense that any convergent fully p ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
We investigate the computational power of continuoustime symmetric Hopfield nets. As is well known, such networks have very constrained, Liapunovfunction controlled dynamics. Nevertheless, we show that they are universal and efficient computational devices, in the sense that any convergent fully parallel computation by a network of n discretetime binary neurons, with in general asymmetric interconnections, can be simulated by a symmetric continuoustime Hopfield net containing only 14n + 6 units using the saturatedlinear sigmoid activation function. In terms of standard discrete computation models this result implies that any polynomially spacebounded Turing machine can be simulated by a polynomially sizeincreasing sequence of continuoustime Hopfield nets.
ContinuousTime Symmetric Hopfield Nets Are Computationally Universal
"... We establish a fundamental result in the theory of computation by continuoustime dynamical systems, by showing that systems corresponding to so called continuoustime symmetric Hopfield nets are capable of general computation. As is well known, such networks have very constrained, Liapunovfunction ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
We establish a fundamental result in the theory of computation by continuoustime dynamical systems, by showing that systems corresponding to so called continuoustime symmetric Hopfield nets are capable of general computation. As is well known, such networks have very constrained, Liapunovfunction controlled dynamics. Nevertheless, we show that they are universal and efficient computational devices, in the sense that any convergent synchronous fully parallel computation by a recurrent network of n discretetime binary neurons, with in general asymmetric coupling weights, can be simulated by a symmetric continuoustime Hopfield net containing only 18n+7 units employing the saturatedlinear activation function. Moreover, if the asymmetric network has maximum integer weight size w_max and converges in discrete time t*, then the corresponding Hopfield net can be designed to operate in continuous time Θ(t*/ε), for any ε > 0...
FiniteState Computation in Analog Neural Networks: Steps Towards Biologically Plausible Models?
, 2001
"... Finitestate machines are the most pervasive models of computation, not only in theoretical computer science, but also in all of its applications to reallife problems, and constitute the best characterized computational model. On the other hand, neural networks proposed almost sixty years ag ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Finitestate machines are the most pervasive models of computation, not only in theoretical computer science, but also in all of its applications to reallife problems, and constitute the best characterized computational model. On the other hand, neural networks proposed almost sixty years ago by McCulloch and Pitts as a simplified model of nervous activity in living beings have evolved into a great variety of socalled artificial neural networks. Artificial neural networks have become a very successful tool for modelling and problem solving because of their builtin learning capability, but most of the progress in this field has occurred with models that are very removed from the behaviour of real, i.e., biological neural networks. This paper surveys the work that has established a connection between finitestate machines and (mainly discretetime recurrent) neural networks, and suggests possible ways to construct finitestate models in biologically plausible neural networks.
Theory of Neuromata
, 1998
"... A finite automaton  the socalled neuromaton, realized by a finite discrete recurrent neural network, working in parallel computation mode, is considered. Both the size of neuromata (i.e., the number of neurons) and their descriptional complexity (i.e., the number of bits in the neuromaton repres ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
A finite automaton  the socalled neuromaton, realized by a finite discrete recurrent neural network, working in parallel computation mode, is considered. Both the size of neuromata (i.e., the number of neurons) and their descriptional complexity (i.e., the number of bits in the neuromaton representation) are studied. It is proved that a constant time delay of the neuromaton output does not play a role within a polynomial descriptional complexity. It is shown that any regular language given by a regular expression of length n is recognized by a neuromaton with \Theta(n) neurons. Further, it is proved that this network size is, in the worst case, optimal. On the other hand, generally there is not an equivalent polynomial length regular expression for a given neuromaton. Then, two specialized constructions of neural acceptors of the optimal descriptional complexity \Theta(n) for a single nbit string recognition are described. They both require O(n 1 2 ) neurons and either O(n) con...
Analog Stable Simulation of Discrete Neural Networks
, 1997
"... The finite discretetime recurrent neural networks are also exploited for potentially infinite computations (e.g. finite automata) where the input is being gradually presented from an external environment via input neurons. Because of gradient learning heuristics or analog hardware implementation re ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
The finite discretetime recurrent neural networks are also exploited for potentially infinite computations (e.g. finite automata) where the input is being gradually presented from an external environment via input neurons. Because of gradient learning heuristics or analog hardware implementation reasons the usage of some continuous activation function is sometimes preferred rather than the discrete hard limiter (threshold function). However, in such cases the approximate representation of finite automaton states by analog network states can lead to an unstable behavior for long input sequences and consequently, to an incorrect resulting computation. Therefore, a stable simulation of any discrete neural network by an analog network of the same size is proposed. The simulation works in real time (`step per step') for any real activation function with different finite limits in improper points. In fact, only the weight parameters of the analog neural network are adjusted to achieve suffi...
EnergyBased Computation with Symmetric Hopfield Nets
"... We propose a unifying approach to the analysis of computational aspects of symmetric Hopfield nets which is based on the concept of "energy source". Within this framework we present different results concerning the computational power of various Hopfield model classes. It is shown that polynomial ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We propose a unifying approach to the analysis of computational aspects of symmetric Hopfield nets which is based on the concept of "energy source". Within this framework we present different results concerning the computational power of various Hopfield model classes. It is shown that polynomialtime computations by nondeterministic Turing machines can be reduced to the process of minimizing the energy in Hopfield nets (the MIN ENERGY problem). Furthermore, external and internal sources of energy are distinguished. The external sources include e.g. energizing inputs from socalled Hopfield languages, and also certain external oscillators that prove finite analog Hopfield nets to be computationally Turing universal. On the other hand, the internal source of energy can be implemented by a symmetric clock subnetwork producing an exponential number of oscillations which are used to energize the simulation of convergent asymmetric networks by Hopfield nets. This shows that infinite families of polynomialsize Hopfield nets compute the complexity class PSPACE/poly. A special attention is paid to generalizing these results for analog states and continuous time to point out alternative sources of efficient computation. 1