## For Neural Networks, Function Determines Form (1992)

### Cached

### Download Links

Citations: | 32 - 14 self |

### BibTeX

@MISC{Albertini92forneural,

author = {Francesca Albertini and Eduardo D. Sontag},

title = {For Neural Networks, Function Determines Form},

year = {1992}

}

### Years of Citing Articles

### OpenURL

### Abstract

This paper shows that the weights of continuous-time feedback neural networks are uniquely identifiable from input/output measurements. Under very weak genericity assumptions, the following is true: Assume given two nets, whose neurons all have the same nonlinear activation function oe; if the two nets have equal behaviors as "black boxes" then necessarily they must have the same number of neurons and ---except at most for sign reversals at each node--- the same weights. Moreover, even if the activations are not a priori known to coincide, they are shown to be also essentially determined from the external measurements. Key words: Neural networks, identification from input/output data, control systems 1 Introduction Many recent papers have explored the computational and dynamical properties of systems of interconnected "neurons." For instance, Hopfield ([7]), Cowan ([4]), and Grossberg and his school (see e.g. [3]), have all studied devices that can be modelled by sets of nonlinear dif...

### Citations

610 |
Neurons with graded response have collective computational properties like those of two-state neurons
- Hopfield
- 1984
(Show Context)
Citation Context ... reversals at each node--- the same weights. 1 Introduction Many recent papers have explored the computational and dynamical properties of systems of interconnected "neurons." For instance, =-=Hopfield ([6]-=-), Cowan ([3]), and Grossberg and his school (see e.g. [2]), have all studied devices that can be modelled by sets of nonlinear differential equations such assx i (t) = 0x i (t) + oe 0 @ n X j=1 a ij ... |

343 | Mathematical Control Theory: Deterministic Finite Dimensional Systems
- Sontag
- 1998
(Show Context)
Citation Context ... is to decide to what extent are the matrices A; B; C determined by the i/o mappings(A;B;C) . In the very special case when oe is the identity, classical linear realization theory ---see for instance =-=[16]-=-, Chapter 5--- implies that, generically, the triple (A; B; C) is determined only up to an invertible change of variables in the state space. That is, except for degenerate situations that arise due t... |

285 |
der Schaft. Nonlinear Dynamical Control Systems
- Njmeijer, van
- 1990
(Show Context)
Citation Context ... input/output map. In this sense, structure (weights) is uniquely determined by function (desired i/o behavior). 1.4 Remarks Note that nonlinear realization theory , as described for instance in [7], =-=[9]-=-, [18], can be also applied to the problem considered here. This theory would allow us to conclude that, under suitable assumptions of controllability and observability, there is some abstract diffeom... |

256 |
Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks
- Cohen, Grossberg
- 1983
(Show Context)
Citation Context ...n Many recent papers have explored the computational and dynamical properties of systems of interconnected "neurons." For instance, Hopfield ([6]), Cowan ([3]), and Grossberg and his school =-=(see e.g. [2]-=-), have all studied devices that can be modelled by sets of nonlinear differential equations such assx i (t) = 0x i (t) + oe 0 @ n X j=1 a ij x j (t) + m X j=1 b ij u j (t) 1 A ; i = 1; : : : ; n ; (1... |

174 |
Nonlinear Control Systems: An Introduction
- Isidori
- 1985
(Show Context)
Citation Context ...sired input/output map. In this sense, structure (weights) is uniquely determined by function (desired i/o behavior). 1.4 Remarks Note that nonlinear realization theory , as described for instance in =-=[7]-=-, [9], [18], can be also applied to the problem considered here. This theory would allow us to conclude that, under suitable assumptions of controllability and observability, there is some abstract di... |

161 | On the computational power of neural nets
- Siegelman, Sontag
- 1995
(Show Context)
Citation Context ...ems that mix linear dynamics and sign functions.) In discrete-time, systems of the type (7) have been recently shown to be at least as powerful as any possible digital computational device (see [12], =-=[13]-=-) when all weights are rational numbers, and a general model of analog computers when the weights are allowed to be real ([14]). 1.1 Uniqueness of Weights Stability properties, memory capacity, and ot... |

153 |
Finite state automata and simple recurrent networks
- Cleeremans, Schreiber, et al.
- 1989
(Show Context)
Citation Context ...odels instead of a real plant, for purposes of control, or for predictive purposes.) This is done, for instance, in certain approaches to grammatical inference and speech processing; see for instance =-=[1]-=-, [10]. Typically, gradient descent algorithms are used in order to fit parameters thorugh the minimization of an error functional that penalizes mismatches between the desired outputs and those that ... |

94 |
Theory of the back-propagation neural network
- Hecht-Nielsen
- 1989
(Show Context)
Citation Context ...ut restricted to the particular case of feedforward (that is, nondynamic) nets, the question of deciding if the only possible symmetries are indeed the ones that we find was asked by Hecht-Nielsen in =-=[4]. The ques-=-tion was partially answered (for so-called "single-hidden layer" nets, and using a particular activation function) by Sussmann in [19], who established a uniqueness result which, in our sett... |

63 | Turing computability with neural nets
- Siegelmann, Sontag
- 1991
(Show Context)
Citation Context ...n systems that mix linear dynamics and sign functions.) In discrete-time, systems of the type (7) have been recently shown to be at least as powerful as any possible digital computational device (see =-=[12]-=-, [13]) when all weights are rational numbers, and a general model of analog computers when the weights are allowed to be real ([14]). 1.1 Uniqueness of Weights Stability properties, memory capacity, ... |

62 |
Nonlinear Dynamical Control Systems
- Nijmeijer, J
- 1990
(Show Context)
Citation Context ...d input/output map. In this sense, structure (weights) is uniquely determined by function (desired i/o behavior). 1.4 Remarks Note that nonlinear realization theory, as described for instance in [8], =-=[10]-=-, [19], can be also applied to the problem considered here. This theory would allow us to conclude that, under suitable assumptions of controllability and observability, there is some abstract diffeom... |

53 | Uniqueness of the weights for minimal feedforward nets with a given input-output map
- Sussmann
- 1992
(Show Context)
Citation Context ...deed the ones that we find was asked by Hecht-Nielsen in [4]. The question was partially answered (for so-called "single-hidden layer" nets, and using a particular activation function) by Su=-=ssmann in [19], who esta-=-blished a uniqueness result which, in our setting, would apply to systems of the special typesx = ~oe(Bu); y = Cx, with oe = tanh(x). (That is, there is no "A" matrix; the result does allow ... |

35 |
Static and dynamic error propagation networks with application to speech coding
- Robinson, Fallside
- 1987
(Show Context)
Citation Context ... instead of a real plant, for purposes of control, or for predictive purposes.) This is done, for instance, in certain approaches to grammatical inference and speech processing; see for instance [1], =-=[10]-=-. Typically, gradient descent algorithms are used in order to fit parameters thorugh the minimization of an error functional that penalizes mismatches between the desired outputs and those that a cand... |

27 |
Analog computation, neural networks, and circuits
- Siegelmann, Sontag
- 1994
(Show Context)
Citation Context ...least as powerful as any possible digital computational device (see [12], [13]) when all weights are rational numbers, and a general model of analog computers when the weights are allowed to be real (=-=[14]-=-). 1.1 Uniqueness of Weights Stability properties, memory capacity, and other characteristics of the above types of systems have been thoroughly investigated by many authors; see for example [5], [8],... |

26 |
Single-input observability of continuous-time systems
- Sussmann
- 1979
(Show Context)
Citation Context ...tically, for determination of the number of units and unique identification of all weights.) The proof of the above Theorem is immediate from the general results for control systems given in [15] and =-=[17], which im-=-ply that identifiability is equivalent to "single experiment" identifiability, for systems defined by analytic differential equations and depending analytically on parameters (here, the weig... |

22 |
Qualitative analysis of neural networks
- Michel, Farrel, et al.
- 1989
(Show Context)
Citation Context ...[14]). 1.1 Uniqueness of Weights Stability properties, memory capacity, and other characteristics of the above types of systems have been thoroughly investigated by many authors; see for example [5], =-=[8], and refe-=-rences there. In this paper, we are interested in studying a somewhat different issue, namely: To what extent does the function of the net, that is to say, the "black box" behavior mapping e... |

21 |
Existence and Uniqueness of Minimal Realizations of Nonlinear Systems
- Sussmann
- 1976
(Show Context)
Citation Context ...t/output map. In this sense, structure (weights) is uniquely determined by function (desired i/o behavior). 1.4 Remarks Note that nonlinear realization theory , as described for instance in [7], [9], =-=[18]-=-, can be also applied to the problem considered here. This theory would allow us to conclude that, under suitable assumptions of controllability and observability, there is some abstract diffeomorphis... |

12 |
On the observability of polynomial systems
- Sontag
- 1979
(Show Context)
Citation Context ...st theoretically, for determination of the number of units and unique identification of all weights.) The proof of the above Theorem is immediate from the general results for control systems given in =-=[15] and [17],-=- which imply that identifiability is equivalent to "single experiment" identifiability, for systems defined by analytic differential equations and depending analytically on parameters (here,... |

9 |
Neural nets
- Cowan, Sharp
- 1988
(Show Context)
Citation Context ... each node--- the same weights. 1 Introduction Many recent papers have explored the computational and dynamical properties of systems of interconnected "neurons." For instance, Hopfield ([6]=-=), Cowan ([3]-=-), and Grossberg and his school (see e.g. [2]), have all studied devices that can be modelled by sets of nonlinear differential equations such assx i (t) = 0x i (t) + oe 0 @ n X j=1 a ij x j (t) + m X... |

6 | Identifiability of discrete-time neural networks - Albertini, Sontag - 1993 |

4 |
Convergent activation dynamics in continuous-time neural networks
- Hirsch
- 1989
(Show Context)
Citation Context ...eal ([14]). 1.1 Uniqueness of Weights Stability properties, memory capacity, and other characteristics of the above types of systems have been thoroughly investigated by many authors; see for example =-=[5], [8], and-=- references there. In this paper, we are interested in studying a somewhat different issue, namely: To what extent does the function of the net, that is to say, the "black box" behavior mapp... |

3 |
Algebraic theory of sign-linear systems
- Schwarzschild, Sontag
- 1991
(Show Context)
Citation Context ...ar systems than is the case for general nonlinear smooth systems. Moreover, for suitably sharp nonlinearities oe, they are approximate models of discontinuous equations such assx = sign(Ax+ Bu). (See =-=[11]-=- for related work on systems that mix linear dynamics and sign functions.) In discrete-time, systems of the type (7) have been recently shown to be at least as powerful as any possible digital computa... |