## Analog Computation with Dynamical Systems (1997)

Venue: | Physica D |

Citations: | 21 - 0 self |

### BibTeX

@ARTICLE{Siegelmann97analogcomputation,

author = {Hava T. Siegelmann and Shmuel Fishman},

title = {Analog Computation with Dynamical Systems},

journal = {Physica D},

year = {1997},

volume = {120},

pages = {120--214}

}

### OpenURL

### Abstract

This paper presents a theory that enables to interpret natural processes as special purpose analog computers. Since physical systems are naturally described in continuous time, a definition of computational complexity for continuous time systems is required. In analogy with the classical discrete theory we develop fundamentals of computational complexity for dynamical systems, discrete or continuous in time, on the basis of an intrinsic time scale of the system. Dissipative dynamical systems are classified into the computational complexity classes P d , Co-RP d , NP d

### Citations

2340 | Computational Complexity - Papadimitriou - 1994 |

1773 |
Introduction to the Theory of Neural Computation
- Hertz, Krogh, et al.
- 1991
(Show Context)
Citation Context ...e Hopfield neural network is of particular interest because it is easily related to other NP-complete problems, and because it provides a natural interface between discrete and continuous computation =-=[15,16]-=-. Our theory is, to some extent, a continuation of their work, in that it provides a natural framework for the complexity analysis of continuous time algorithms. Furthermore, we allow for attractors w... |

1574 |
Neural networks and physical systems with emergent collective computational abilities
- Hopfield
- 1982
(Show Context)
Citation Context ...e Hopfield neural network is of particular interest because it is easily related to other NP-complete problems, and because it provides a natural interface between discrete and continuous computation =-=[15,16]-=-. Our theory is, to some extent, a continuation of their work, in that it provides a natural framework for the complexity analysis of continuous time algorithms. Furthermore, we allow for attractors w... |

1328 |
Nonlinear oscillations, dynamical systems, and bifurcations of vector fields
- Guckenheimer, Holmes
- 1983
(Show Context)
Citation Context ... �� will converge to the same attractor as the corresponding continuous system. Therefore the results of the computation may be different for the two systems. Hyperbolic attractors, on the other h=-=and [2]-=-, remain stable under small perturbations and therefore the map (4.21) and the corresponding continuous systems lead to the same computations, provided the initial conditions are sufficiently far from... |

692 | E.: Grounding in communication
- Clark, Brennan
- 1991
(Show Context)
Citation Context ...e conjecture that usually (in the sense of prevalence) the sets of points on the attractor for which the Lyapunov exponent is nonnegative in directions transverse to the attractor, is of measure zero =-=[69]-=-. (For the discussion of transverse Lyapunov exponents see for example [70].) As for the type of attractors, it is believed that systems with chaotic attractors are neither prevalent nor shy, but we a... |

590 |
Neurons with graded response have collective computational properties like those of two-state neurons
- Hopfield
- 1984
(Show Context)
Citation Context ...ative dynamical systems for which an energy (Lyapunov) functional is defined and patterns reside in the location of local minima of the energy. The most popular model is probably the Hopfield network =-=[54,55,16]-=-. Other related models are contained in the works by Amari, Anderson, Kohonen, Nakano, Willshaw, Little, Atiya and Abu-Mostafa, and Morita [56--66]. The meaningful attractors of these networks --- whe... |

482 |
Neural computation of decisions on optimization problems
- Hopfield, Tank
- 1985
(Show Context)
Citation Context ...ative dynamical systems for which an energy (Lyapunov) functional is defined and patterns reside in the location of local minima of the energy. The most popular model is probably the Hopfield network =-=[54,55,16]-=-. Other related models are contained in the works by Amari, Anderson, Kohonen, Nakano, Willshaw, Little, Atiya and Abu-Mostafa, and Morita [56--66]. The meaningful attractors of these networks --- whe... |

468 |
The Emperor’s New Mind
- Penrose
- 1989
(Show Context)
Citation Context ... memory (see section 2.4). Our theory can be used to analyze some of these models in terms of computational complexity. Attractor models of awareness, feeling and related activities of the brain (see =-=[18]) are beyo-=-nd the scope of this paper. Third, analog computation can be utilized to test possible theoretical limitations of the "physical Church-Turing thesis" [19] that states that the computational ... |

377 |
On a theory of computation and complexity over the real numbers
- Blum, Shub, et al.
- 1989
(Show Context)
Citation Context ... section with a mention of relevant work in the field of analog neurodynamics. 2.1 Discrete time models Two well known analog computation models are the BSS model of computation over the real numbers =-=[23]-=- and the SiSo analog recurrent neural network [24,21]. Smale was the first to insist on a computational model, in which the operations are done on the values irrespectively of their radix two represen... |

353 |
Chaos in Dynamical Systems
- Ott
- 2002
(Show Context)
Citation Context ... properties of their attractors. The convergence to the attractors is usually exponential. The complexity of an attractor is quantified by its Kolmogorov-Sinai (KS) entropy and its Lyapunov exponents =-=[1,5,6]-=-. These essentially measure the sensitivity to changes in initial conditions, and therefore can be viewed as a measure of the degree of chaos. For stable fixed points and limit cycles the largest Lyap... |

321 | Shadows of the Mind
- Penrose
- 1994
(Show Context)
Citation Context ...theoretical limitations of the "physical Church-Turing thesis" [19] that states that the computational capabilities of any physical device should not exceed (in idealization) that of a Turin=-=g machine [20]. If a dev-=-ice that computes problems that cannot be computed by the Turing machine (and therefore digital computers) is found, it will challenge the "physical Church-Turing thesis" and will therefore ... |

222 |
Quantum Mechanical Computers
- Feynman
- 1985
(Show Context)
Citation Context ...elated activities of the brain (see [18]) are beyond the scope of this paper. Third, analog computation can be utilized to test possible theoretical limitations of the "physical Church-Turing the=-=sis" [19]-=- that states that the computational capabilities of any physical device should not exceed (in idealization) that of a Turing machine [20]. If a device that computes problems that cannot be computed by... |

219 |
Ergodic Problems of Classical Mechanics
- Arnol’d, Avez
- 1968
(Show Context)
Citation Context ...rate. 4 Maps versus continuous time systems Continuous time systems are natural for the description of the dynamics of physical objects, but there are situations where the dynamics is modeled by maps =-=[1,3,4,6]-=-. Therefore the scope of the theory should be extended to include maps. Furthermore, the inclusion of maps facilitates a relatively easy comparison with the concepts of standard digital computation. M... |

183 |
Cellular Automata Machines
- Toffoli, Margolus
- 1987
(Show Context)
Citation Context ...etwork. As a result even a finite coupled map lattice is computationally stronger than a universal Turing machine. CA and coupled map lattices are used in modeling a broad class of physical phenomena =-=[33]-=-. Coupled map lattices can be considered time and space discretizations of PDE's, which are thus computationally universal as well [34]. 2.2 Hybrid models Motivation for some of the work on hybrid com... |

178 |
Differential Equations, Dynamical Systems and Linear Algebra
- Hirsch, Smale
- 1974
(Show Context)
Citation Context ...differentiable functional E(x) satisfying dE(x(t)) dt ! 0; (5.1) along every trajectory x(t), except points x i where this derivative vanishes. If the x i 's are isolated then by the Lyapunov theorem =-=[2,76] these f-=-ixed points are the attractors of the system, if there is a neighborhood of x i where E is larger than E(x i ). In this case basin boundaries are smooth. Discretizing with small �� 1 �� [E(x n... |

140 | Optimization and dynamical systems
- Helmke, Moore
- 1994
(Show Context)
Citation Context ...n by a number of researchers. Brockett introduced a set of ODE's that perform various tasks such as sorting and solving linear programming problems [13]. In the comprehensive book of Helmke and Moore =-=[14]-=- one can find numerous other applications and references, among them, the state of the art in dynamical systems for linear programming. The Hopfield neural network is of particular interest because it... |

139 |
Hybrid models for motion control systems
- Brockett
- 1993
(Show Context)
Citation Context ...Kohn [37], Brockett [38--40], and Branicky [41,42]; more can be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as in "stepper motor" and in tra=-=nsmission [40]-=- as well as their stabilizing properties [46]. 2.3 Continuous time models A fundamental question is the computability with differential equations. ODE's were used to simulate various discrete time mod... |

136 |
Stable and Random Motions in Dynamical Systems
- Moser
- 2001
(Show Context)
Citation Context .... If ! is rational the trajectory is periodic, and it is quasi-periodic for irrational !. If some disturbance is added so that the system is not integrable anymore, by Kolmogorov-Arnold-Moser theorem =-=[2--4,82,83]-=- most of the irrational trajectories around the elliptic fixed point will be only slightly deformed; by the Poincar'e-Birkhoff theorem [2--4,84], the rational trajectories will be replaced by a sequen... |

128 | Quantitative universality for a class of nonlinear transformations
- Feigenbaum
(Show Context)
Citation Context ...nges of parameters. Since �� can be treated as a parameter, even in the limit �� ! 0, the attractors for �� 6= 0 may be different from these at �� = 0. This is the situation even for t=-=he logistic map [73,1]-=- where arbitrarily small perturbations may lead to the replacement of the chaotic attractor by a periodic orbit. Consequently one cannot be assured that a map of the form (4.21) taken to any arbitrary... |

128 |
Fractal measures and their singularities: the characterization of strange sets, Phys
- Halsey, Jensen, et al.
- 1986
(Show Context)
Citation Context ...onentially long time in the second case. A typical strange attractor is multifractal: its measure on various points is not uniform. Let the measure in a hyper-sphere of radius ffl on the attractor be =-=[1,86] �-=-� i = ffl ff i : (6.1) The measure is smallest in regions where ff i = ff max . The time it takes to reach the region of the smallest measure is inversely proportional to it. Therefore, the time req... |

108 |
On the nature of turbulence
- Ruelle, Takens
- 1971
(Show Context)
Citation Context ...le or chaotic) as long as exponential 1 For functions F that are sufficiently smooth, in each vicinity (in function space) of the trivial identity F(x) = 0, there are flows x(t) that are chaotic, see =-=[71,72]-=-. Multiplying F by a constant a indeed does not affect the complexity of the function, but the limit a ! 0 should not be taken since in this limit the system is clearly regular. convergence is assured... |

99 |
Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl
- Brockett
- 1991
(Show Context)
Citation Context ...m an initial condition to an attractor has been taken by a number of researchers. Brockett introduced a set of ODE's that perform various tasks such as sorting and solving linear programming problems =-=[13]-=-. In the comprehensive book of Helmke and Moore [14] one can find numerous other applications and references, among them, the state of the art in dynamical systems for linear programming. The Hopfield... |

98 | Non-holographic associative memory - Willshaw, Buneman, et al. - 1969 |

97 |
Models for hybrid systems: Automata, topologies, stability,” in Grossman et al
- Nerode, Kohn
(Show Context)
Citation Context ...screte and continuous dynamics, usually represented by ODE's that are governed by finite automata. Among famous hybrid models are the works by Tavernini [35], Back-GuckenheimerMyers [36], Nerode-Kohn =-=[37], Brockett-=- [38--40], and Branicky [41,42]; more can be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as in "stepper motor" and in transmission [40] as w... |

89 |
Unpredictability and undecidability in dynamical systems
- Moore
- 1990
(Show Context)
Citation Context ... those of the classical standard digital computer (superTuring) . Some work was done on comparing these models and on interleaving them [25--27]. The generalized shift map (GS) was discussed by Moore =-=[28]-=-. He shows it to be computationally universal, and claims that this model is physically realizable. This is in contrast to his model [29], which he views as unphysical, and his dynamical recognizers [... |

88 | Correlation Matrix Memories - Kohonen - 1972 |

87 | Analog computation via neural networks
- Siegelmann, Sontag
- 1994
(Show Context)
Citation Context ..., it will challenge the "physical Church-Turing thesis" and will therefore be of great interest. Some theoretical analog models of computation have the capability of computing beyond the Tur=-=ing limit [21,22]-=-, but no realizable super-Turing system has been noted. We do not suggest the current work as providing a step towards the identification of super-Turing natural systems. We rather concentrate on perc... |

79 |
Small denominators and problems of stability of motion in classical and celestial mechanics
- Arnold
- 1963
(Show Context)
Citation Context .... If ! is rational the trajectory is periodic, and it is quasi-periodic for irrational !. If some disturbance is added so that the system is not integrable anymore, by Kolmogorov-Arnold-Moser theorem =-=[2--4,82,83]-=- most of the irrational trajectories around the elliptic fixed point will be only slightly deformed; by the Poincar'e-Birkhoff theorem [2--4,84], the rational trajectories will be replaced by a sequen... |

74 |
Chaos and integrability in nonlinear dynamics
- Tabor
- 1989
(Show Context)
Citation Context ...rate. 4 Maps versus continuous time systems Continuous time systems are natural for the description of the dynamics of physical objects, but there are situations where the dynamics is modeled by maps =-=[1,3,4,6]-=-. Therefore the scope of the theory should be extended to include maps. Furthermore, the inclusion of maps facilitates a relatively easy comparison with the concepts of standard digital computation. M... |

73 | Recursion theory on the reals and continuous-time computation, Theoret
- Moore
- 1996
(Show Context)
Citation Context ...25--27]. The generalized shift map (GS) was discussed by Moore [28]. He shows it to be computationally universal, and claims that this model is physically realizable. This is in contrast to his model =-=[29]-=-, which he views as unphysical, and his dynamical recognizers [30]. An extension of the GS to include real constants was suggested in [22]. This analog version of the GS has super-Turing computational... |

72 |
Deterministic chaos
- Schuster
- 1988
(Show Context)
Citation Context ... properties of their attractors. The convergence to the attractors is usually exponential. The complexity of an attractor is quantified by its Kolmogorov-Sinai (KS) entropy and its Lyapunov exponents =-=[1,5,6]-=-. These essentially measure the sensitivity to changes in initial conditions, and therefore can be viewed as a measure of the degree of chaos. For stable fixed points and limit cycles the largest Lyap... |

68 | Universal computation and other capabilities of hybrid and continuous dynamical systems. Theoret
- Branicky
- 1995
(Show Context)
Citation Context ...y represented by ODE's that are governed by finite automata. Among famous hybrid models are the works by Tavernini [35], Back-GuckenheimerMyers [36], Nerode-Kohn [37], Brockett [38--40], and Branicky =-=[41,42]; more can-=- be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as in "stepper motor" and in transmission [40] as well as their stabilizing properties [46].... |

66 |
Abstract computability and its relations to the general purpose analog computer
- Pour-El
- 1974
(Show Context)
Citation Context ...lems solvable by dynamical systems are shown in the book of Helmke and Moore [14]. Another line of work regarded the so called "general purpose analog computer" was dominated by Shannon [48]=-=, Pour-el [49] and Rubel-=- [50--52]. Despite the similar title, that "analog computer" is very different from our approach. It describes a mathematical abstraction that consists of finite numbers of boxes with plenty... |

65 |
Computability in analysis and physics
- Richards
- 1989
(Show Context)
Citation Context ...roblem and generating the Euler's gamma function); their general purpose analog computer produces only solutions of initial-value problems for algebraic ordinary differential equations [52]. The book =-=[53]-=- describes a particular interesting view of computability in analysis, differential equations, and Banach spaces. The important questions raised by these seminal works inspired many researchers, inclu... |

63 |
Computation beyond the Turing limit
- Siegelmann
- 1995
(Show Context)
Citation Context ..., it will challenge the "physical Church-Turing thesis" and will therefore be of great interest. Some theoretical analog models of computation have the capability of computing beyond the Tur=-=ing limit [21,22]-=-, but no realizable super-Turing system has been noted. We do not suggest the current work as providing a step towards the identification of super-Turing natural systems. We rather concentrate on perc... |

63 | A simple neural network generating an interactive memory - Anderson - 1972 |

56 | Dynamical recognizers: Real-time language recognition by analog computers
- Moore
- 1998
(Show Context)
Citation Context ...]. He shows it to be computationally universal, and claims that this model is physically realizable. This is in contrast to his model [29], which he views as unphysical, and his dynamical recognizers =-=[30]-=-. An extension of the GS to include real constants was suggested in [22]. This analog version of the GS has super-Turing computational power as well. Cellular automata (CA) are a computational model w... |

53 |
On the computational power of neural networks
- Siegelmann, Sontag
- 1995
(Show Context)
Citation Context ...eld of analog neurodynamics. 2.1 Discrete time models Two well known analog computation models are the BSS model of computation over the real numbers [23] and the SiSo analog recurrent neural network =-=[24,21]-=-. Smale was the first to insist on a computational model, in which the operations are done on the values irrespectively of their radix two representation. Together with Blum and Shub they introduced a... |

53 |
Differential automata and their discrete simulators
- Tavernini
- 1987
(Show Context)
Citation Context ...of discrete dynamics. Hybrid systems combine discrete and continuous dynamics, usually represented by ODE's that are governed by finite automata. Among famous hybrid models are the works by Tavernini =-=[35]-=-, Back-GuckenheimerMyers [36], Nerode-Kohn [37], Brockett [38--40], and Branicky [41,42]; more can be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as i... |

51 |
Mathematical theory of the differential analyzer
- Shannon
- 1941
(Show Context)
Citation Context ...ous other problems solvable by dynamical systems are shown in the book of Helmke and Moore [14]. Another line of work regarded the so called "general purpose analog computer" was dominated b=-=y Shannon [48], Pour-el -=-[49] and Rubel [50--52]. Despite the similar title, that "analog computer" is very different from our approach. It describes a mathematical abstraction that consists of finite numbers of box... |

49 |
From attractor to chaotic saddle: a tale of transverse instability
- Ashwin, Buescu, et al.
- 1996
(Show Context)
Citation Context ...n the attractor for which the Lyapunov exponent is nonnegative in directions transverse to the attractor, is of measure zero [69]. (For the discussion of transverse Lyapunov exponents see for example =-=[70]-=-.) As for the type of attractors, it is believed that systems with chaotic attractors are neither prevalent nor shy, but we are not aware of any decisive relevant rigorous result; our theory allows fo... |

43 |
Order within Chaos
- Berge, Pomeau, et al.
- 1984
(Show Context)
Citation Context ... properties of their attractors. The convergence to the attractors is usually exponential. The complexity of an attractor is quantified by its Kolmogorov-Sinai (KS) entropy and its Lyapunov exponents =-=[1,5,6]-=-. These essentially measure the sensitivity to changes in initial conditions, and therefore can be viewed as a measure of the degree of chaos. For stable fixed points and limit cycles the largest Lyap... |

42 |
Simple computation-universal cellular spaces
- Smith
- 1971
(Show Context)
Citation Context ...computational model which is an infinite lattice of discrete variables with a local homogeneous transition rule. CA's contain Turing machines as a special case, and are thus computationally universal =-=[31]-=-. When the variables are reals the machine is sometimes called an analog cellular automaton, or a coupled map lattice (CML) [32]. It can be thought of as a generalization of both BSS and SiSo models. ... |

41 |
The extended analog computer
- Rubel
- 1993
(Show Context)
Citation Context ...the Dirichlet problem and generating the Euler's gamma function); their general purpose analog computer produces only solutions of initial-value problems for algebraic ordinary differential equations =-=[52]-=-. The book [53] describes a particular interesting view of computability in analysis, differential equations, and Banach spaces. The important questions raised by these seminal works inspired many res... |

38 |
A dynamical simulation facility for hybrid systems
- Back, Guckenheimer, et al.
- 1993
(Show Context)
Citation Context ...systems combine discrete and continuous dynamics, usually represented by ODE's that are governed by finite automata. Among famous hybrid models are the works by Tavernini [35], Back-GuckenheimerMyers =-=[36], Nerode-K-=-ohn [37], Brockett [38--40], and Branicky [41,42]; more can be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as in "stepper motor" and in tran... |

32 | Associative memory with nonmonotone dynamics - Morita - 1993 |

31 | O.: On Some Relations Between Dynamical Systems and Transition Systems - ASARIN |

30 | A weak version of the Blum, Shub & Smale model - Koiran - 1993 |

30 | Smooth dynamical systems which realize arithmetical and logical operations, in: H - Brockett - 1989 |

30 | Topology of hybrid systems
- Branicky
- 1993
(Show Context)
Citation Context ...y represented by ODE's that are governed by finite automata. Among famous hybrid models are the works by Tavernini [35], Back-GuckenheimerMyers [36], Nerode-Kohn [37], Brockett [38--40], and Branicky =-=[41,42]; more can-=- be found in [43--45]. The main interest in hybrid systems stems from their practicality --- such as in "stepper motor" and in transmission [40] as well as their stabilizing properties [46].... |