Motivated partly by the resurgence of neural computation research, and partly by advances in device technology, there has been a recent increase of interest in analog, continuous-time computation. However, while special-case algorithms and devices are being developed, relatively little work exists on the general theory of continuous-time models of computation. In this paper, we survey the existing models and results in this area, and point to some of the open research questions. 1 Introduction After a long period of oblivion, interest in analog computation is again on the rise. The immediate cause for this new wave of activity is surely the success of the neural networks "revolution", which has provided hardware designers with several new numerically based, computationally interesting models that are structurally sufficiently simple to be implemented directly in silicon. (For designs and actual implementations of neural models in VLSI, see e.g. [30, 45]). However, the more fundamental...

We investigate the computational power of continuous-time neural networks with Hopfield-type units. We prove that polynomial-size networks with saturated-linear response functions are at least as powerful as polynomially space-bounded Turing machines. 1 Introduction In a paper published in 1984 [11], John Hopfield introduced a continuoustime version of the neural network model whose discrete-time variant he had discussed in his seminal 1982 paper [10]. The 1984 paper also contains an electronic implementation scheme for the continuous-time networks, and an argument showing that for sufficiently large-gain nonlinearities, these behave similarly to the discrete-time ones, at least when used as associative memories. The power of Hopfield's discrete-time networks as general-purpose computational devices was analyzed in [17, 18]. In this paper we conduct a similar analysis for networks consisting of Hopfield's continuous-time units; however we are at this stage able to analyze only the gen...

INTRODUCTION The two main streams of neural networks research consider neural networks either as a powerful family of nonlinear statistical models, to be used in for example pattern recognition applications [6], or as formal models to help develop a computational understanding of the brain [10]. Historically, the brain theory interest was primary [32], but with the advances in computer technology, the application potential of the statistical modeling techniques has shifted the balance. 1 The study of neural networks as general computational devices does not strictly follow this division of interests: rather, it provides a general framework outlining the limitations and possibilities aecting both research domains. The prime historic example here is obviously Minsky's and Papert's 1969 study of the computational limitations of singlelayer perceptrons [34], which was a major inuence in turning away interest from neural network learning to symbolic AI techniques for more

We establish a fundamental result in the theory of computation by continuous-time dynamical systems, by showing that systems corresponding to so called continuous-time symmetric Hopfield nets are capable of general computation. As is well known, such networks have very constrained, Liapunov-function controlled dynamics. Nevertheless, we show that they are universal and efficient computational devices, in the sense that any convergent synchronous fully parallel computation by a recurrent network of n discrete-time binary neurons, with in general asymmetric coupling weights, can be simulated by a symmetric continuous-time Hopfield net containing only 18n+7 units employing the saturated-linear activation function. Moreover, if the asymmetric network has maximum integer weight size w_max and converges in discrete time t*, then the corresponding Hopfield net can be designed to operate in continuous time Θ(t*/ε), for any ε > 0...

We establish a fundamental result in the theory of computation by continuous-time dynamical systems, by showing that systems corresponding to so called continuous-time symmetric Hopfield nets are capable of general computation. More precisely, we prove that any function computed by a discrete-time asymmetric recurrent network of n threshold gates can also be computed by a continuous-time symmetrically-coupled Hopfield system of dimension 18n + 7. Moreover, if the threshold logic network has maximum weight wmax and converges in discrete time t , then the corresponding Hopfield system can be designed to operate in continuous time (t ="), for any value 0 < " < 0:0025 such that wmax2 3n "2 1=" . The result appears at rst sight counterintuitive, because the dynamics of any symmetric Hopfield system is constrained by a Liapunov, or energy function defined on its state space. In particular, such a system always converges from any initial state towards some stable equilibrium state, and hence cannot exhibit nondamping oscillations, i.e. strictly speaking cannot simulate even a single alternating bit. However, we show that if one only considers terminating computations, then the Liapunov constraint can be overcome, and one can in fact embed arbitrarily complicated computations in the dynamics of Liapunov systems with only a modest cost in the system's dimensionality. In terms of standard discrete computation models, our result implies that any polynomially space-bounded Turing machine can be simulated by a family of polynomial-size continuous-time symmetric Hopfield nets.