this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NP-hard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into two halfspaces, and the dynamics in eachhalfspace correspond to a differentlinear system

Motivated partly by the resurgence of neural computation research, and partly by advances in device technology, there has been a recent increase of interest in analog, continuous-time computation. However, while special-case algorithms and devices are being developed, relatively little work exists on the general theory of continuous-time models of computation. In this paper, we survey the existing models and results in this area, and point to some of the open research questions. 1 Introduction After a long period of oblivion, interest in analog computation is again on the rise. The immediate cause for this new wave of activity is surely the success of the neural networks "revolution", which has provided hardware designers with several new numerically based, computationally interesting models that are structurally sufficiently simple to be implemented directly in silicon. (For designs and actual implementations of neural models in VLSI, see e.g. [30, 45]). However, the more fundamental...

We show closed-form analytic functions consisting of a finite number of trigonometric terms can simulate Turing machines, with exponential slowdown in one dimension or in real time in two or more. 1 A part of this author's work was done when he was visiting DIMACS at Rutgers University. 1 Introduction Various authors have independently shown [9, 12, 4, 14, 1] that finite-dimensional piecewise-linear maps and flows can simulate Turing machines. The construction is simple: associate the digits of the x and y coordinates of a point with the left and right halves of a Turing machine's tape. Then we can shift the tape head by halving or doubling x and y, and write on the tape by adding constants to them. Thus two dimensions suffice for a map, or three for a continuous-time flow. These systems can be thought of as billiards or optical ray tracing in three dimensions, recurrent neural networks, or hybrid systems. However, piecewise-linear functions are not very realistic from a physical p...

We deal with computational issues of loading a fixed-architecture neural network with a set of positive and negative examples. This is the first result on the hardness of loading networks which do not consist of the binary-threshold neurons, but rather utilize a particular continuous activation function, commonly used in the neural network literature. We observe that the loading problem is polynomial-time if the input dimension is constant. Otherwise, however, any possible learning algorithm based on particular fixed architectures faces severe computational barriers. Similar theorems have already been proved by Megiddo and by Blum and Rivest, to the case of binary-threshold networks only. Our theoretical results lend further justification to the use of incremental (architecture-changing) techniques for training networks rather than fixed architectures. Furthermore, they imply hardness of learnability in the probably-approximately-correct sense as well.

Most of the work on the Vapnik-Chervonenkis dimension of neural networks has been focused on feedforward networks. However, recurrent networks are also widely used in learning applications, in particular when time is a relevant parameter. This paper provides lower and upper bounds for the VC dimension of such networks. Several types of activation functions are discussed, including threshold, polynomial, piecewisepolynomial and sigmoidal functions. The bounds depend on two independent parameters: the number w of weights in the network, and the length k of the input sequence. In contrast, for feedforward networks, VC dimension bounds can be expressed as a function of w only. An important difference between recurrent and feedforward nets is that a fixed recurrent net can receive inputs of arbitrary length. Therefore we are particularly interested in the case k AE w. Ignoring multiplicative constants, the main results say roughly the following: ffl For architectures with activation oe = a...

We examine the representational capabilities of first-order and second-order Single Layer Recurrent Neural Networks (SLRNNs) with hard-limiting neurons. We show that a secondorder SLRNN is strictly more powerful than a first-order SLRNN. However, if the first-order SLRNN is augmented with output layers of feedforward neurons, it can implement any finitestate recognizer, but only if state-splitting is employed. When a state is split, it is divided into two equivalent states. The judicious use of state-splitting allows for efficient implementation of finite-state recognizers using augmented first-order SLRNNs.

. We survey some of the central results in the complexity theory of discrete neural networks, with pointers to the literature. Our main emphasis is on the computational power of various acyclic and cyclic network models, but we also discuss briefly the complexity aspects of synthesizing networks from examples of their behavior. CR Classification: F.1.1 [Computation by Abstract Devices]: Models of Computation---neural networks, circuits; F.1.3 [Computation by Abstract Devices ]: Complexity Classes---complexity hierarchies Key words: Neural networks, computational complexity, threshold circuits, associative memory 1. Introduction The currently again very active field of computation by "neural" networks has opened up a wealth of fascinating research topics in the computational complexity analysis of the models considered. While much of the general appeal of the field stems not so much from new computational possibilities, but from the possibility of "learning", or synthesizing networks...

We describe an emerging field, that of nonclassical computability and nonclassical computing machinery. According to the nonclassicist, the set of well-defined computations is not exhausted by the computations that can be carried out by a Turing machine. We provide an overview of the field and a philosophical defence of its foundations.

95 CDC- Keywords: complexity, controllability, nonlinear Extended Summary for Invited Session entitled Computational Complexity Issues in Control 1. Introduction It is obvious that many control problems are in general easier to solve for linear systems than for arbitrary, not necessarily linear, ones. An interesting and worthy area of research deals with the attempt to make mathematically precise the increases in difficulty that may arise when passing to the nonlinear case. By obtaining such precise statements, one gains an understanding of which analysis and/or design problems may be expected to be intractable. For instance, even for apparently mildly nonlinear systems it becomes impossible to check if a state ever reaches the origin. More interestingly perhaps, one also can then explain in what sense some variants of problems are easier than others for nonlinear systems. An example of this later aspect is given by comparing the characterization of the accessibility property (being ...

One of the issues in any learning model is how it scales with problem size. The problem of learning finite state machine (FSMs) from examples with recurrent neural networks has been extensively explored. However, these results are somewhat disappointing in the sense that the machines that can be learned are too small to be competitive with existing grammatical inference algorithms. We show that a type of recurrent neural network (Narendra & Parthasarathy, 1990, IEEE Trans. Neural Networks, 1, 4-27) which has feedback but no hidden state neurons can learn a special type of FSM called a finite memory machine (FMM) under certain constraints. These machines have a large number of states (simulations are for 256 and 512 state FMMs) but have minimal order, relatively small depth and little logic when the FMM is implemented as a sequential machine,