Motivated partly by the resurgence of neural computation research, and partly by advances in device technology, there has been a recent increase of interest in analog, continuous-time computation. However, while special-case algorithms and devices are being developed, relatively little work exists on the general theory of continuous-time models of computation. In this paper, we survey the existing models and results in this area, and point to some of the open research questions. 1 Introduction After a long period of oblivion, interest in analog computation is again on the rise. The immediate cause for this new wave of activity is surely the success of the neural networks "revolution", which has provided hardware designers with several new numerically based, computationally interesting models that are structurally sufficiently simple to be implemented directly in silicon. (For designs and actual implementations of neural models in VLSI, see e.g. [30, 45]). However, the more fundamental...

We investigate the computational power of continuous-time neural networks with Hopfield-type units. We prove that polynomial-size networks with saturated-linear response functions are at least as powerful as polynomially space-bounded Turing machines. 1 Introduction In a paper published in 1984 [11], John Hopfield introduced a continuoustime version of the neural network model whose discrete-time variant he had discussed in his seminal 1982 paper [10]. The 1984 paper also contains an electronic implementation scheme for the continuous-time networks, and an argument showing that for sufficiently large-gain nonlinearities, these behave similarly to the discrete-time ones, at least when used as associative memories. The power of Hopfield's discrete-time networks as general-purpose computational devices was analyzed in [17, 18]. In this paper we conduct a similar analysis for networks consisting of Hopfield's continuous-time units; however we are at this stage able to analyze only the gen...

The finite discrete-time recurrent neural networks are also exploited for potentially infinite computations (e.g. finite automata) where the input is being gradually presented from an external environment via input neurons. Because of gradient learning heuristics or analog hardware implementation reasons the usage of some continuous activation function is sometimes preferred rather than the discrete hard limiter (threshold function). However, in such cases the approximate representation of finite automaton states by analog network states can lead to an unstable behavior for long input sequences and consequently, to an incorrect resulting computation. Therefore, a stable simulation of any discrete neural network by an analog network of the same size is proposed. The simulation works in real time (`step per step') for any real activation function with different finite limits in improper points. In fact, only the weight parameters of the analog neural network are adjusted to achieve suffi...

We propose a unifying approach to the analysis of computational aspects of symmetric Hopfield nets which is based on the concept of "energy source". Within this framework we present different results concerning the computational power of various Hopfield model classes. It is shown that polynomial-time computations by nondeterministic Turing machines can be reduced to the process of minimizing the energy in Hopfield nets (the MIN ENERGY problem). Furthermore, external and internal sources of energy are distinguished. The external sources include e.g. energizing inputs from so-called Hopfield languages, and also certain external oscillators that prove finite analog Hopfield nets to be computationally Turing universal. On the other hand, the internal source of energy can be implemented by a symmetric clock subnetwork producing an exponential number of oscillations which are used to energize the simulation of convergent asymmetric networks by Hopfield nets. This shows that infinite families of polynomial-size Hopfield nets compute the complexity class PSPACE/poly. A special attention is paid to generalizing these results for analog states and continuous time to point out alternative sources of efficient computation. 1

. We describe a simple general purpose condition-action type parallel programming language and its implementation on Hopfield-type neural networks. A prototype compiler performing the translation has been implemented. Keywords: neural networks, Hopfield nets, parallel programming 1 Introduction In addition to their uses in specific applications such as pattern classification, associative memory or combinatorial optimization recurrent neural networks are also theoretically universal, i.e., general-purpose computing devices. It was observed in [7] that sequences of polynomial-size recurrent threshold logic networks are computationally equivalent to (nonuniform) polynomial space-bounded Turing machines, and in [11] this equivalence was extended to polynomial-size sequences of networks with symmetric weights, i.e., "Hopfield nets" with hidden units. A corollary to the latter construction shows also that polynomial-size symmetric networks with small (i.e., polynomially bounded) intercon...

The present paper investigates four relatively independent issues, each in one section, which complete our knowledge regarding the computational aspects of popular Hopfield nets [9]. In Section 2, the computational equivalence of convergent asymmetric and Hopfield nets is shown with respect to the network size. In Section 3, the convergence time of Hopfield nets is analyzed in terms of bit representations. In Section 4, a polynomial time approximate algorithm for the minimum energy problem is shown. In Section 5, the Turing universality of analog Hopfield nets is studied.

We investigate the computational power of continuous-time symmetric Hopfield nets. Since the dynamics of such networks are governed by Liapunov (energy) functions, they cannot generate infinite nondamping oscillations, and hence cannot simulate arbitrary (potentially divergent) discrete computations. Nevertheless, we prove that any convergent fully parallel computation by a network of n discrete-time binary neurons, with in general asymmetric interconnections, can be simulated by a symmetric continuoustime Hopfield net containing 14n+6 units using the saturated-linear sigmoid activation function. In terms of standard discrete computation models this result implies that any polynomially space-bounded Turing machine can be simulated by a polynomially size-increasing sequence of continuous-time Hopfield nets. Similar techniques as here yield corresponding results on the convergence time and computational power of discrete-time Hopfield nets.

This technical report addresses a particular approach to modeling and analysis of the behavior of large-scale multi-agent systems. A broad variety of multi-agent systems are modeled as appropriate variants of cellular and network automata. Several fundamental properties of the collective dynamics of those cellular and network automata are then formally analyzed. Various loosely coupled large-scale distributed information systems are of an increasing interest in a variety of areas of computer science and its applications – areas as diverse as team robotics, intelligent transportation systems, open distributed software environments, disaster response management, distributed databases and information retrieval, and computational theories of language evolution. A popular paradigm for abstracting such distributed infrastructures is that of multi-agent systems (MAS) made of typically a large number of autonomous agents that locally interact with each other. This report is an attempt at a cellular and network automata based mathematical and computational theory of such MAS. The