## Absence of Cycles in Symmetric Neural Networks (1995)

Venue: | Advances in Neural Information Processing Systems (NIPS) 8 |

Citations: | 3 - 1 self |

### BibTeX

@INPROCEEDINGS{Wang95absenceof,

author = {Xin Wang and Arun Jagota and A Botelho and Max Garzon},

title = {Absence of Cycles in Symmetric Neural Networks},

booktitle = {Advances in Neural Information Processing Systems (NIPS) 8},

year = {1995},

pages = {372--378},

publisher = {MIT Press}

}

### OpenURL

### Abstract

For a given recurrent neural network, a discrete-time model may have asymptotic dynamics different from the one of a related continuous-time model. In this paper, we consider a discrete-time model that discretizes the continuous-time leaky integrator model and study its parallel and sequential dynamics for symmetric networks. We provide sufficient (and necessary in many cases) conditions for the discretized model to have the same cycle-free dynamics of the corresponding continuous-time model in symmetric networks. 1 INTRODUCTION For an n-neuron recurrent network, a much-studied and widely-used continuoustime (CT) model is the leaky integrator model (Hertz, et al., 1991; Hopfield, 1984), given by a system of nonlinear differential equations: ø i dx i dt = \Gammax i + oe i ( n X j=1 w ij x j + I i ); t 0; i = 1; :::; n; (1) and a related discrete-time (DT) version is the sigmoidal model (Hopfield, 1982; Marcus & Westervelt, 1989), specified by a system of nonlinear difference e...

### Citations

1896 |
Introduction to the theory of neural computation
- Hertz, Krogh, et al.
- 1991
(Show Context)
Citation Context ...he corresponding continuous-time model in symmetric networks. 1 INTRODUCTION For an n-neuron recurrent network, a much-studied and widely-used continuoustime (CT) model is the leaky integrator model (=-=Hertz, et al., 1991; Ho-=-pfield, 1984), given by a system of nonlinear differential equations: �� i dx i dt = \Gammax i + oe i ( n X j=1 w ij x j + I i ); ts0; i = 1; :::; n; (1) and a related discrete-time (DT) version i... |

625 |
Neuron with graded response have collective computational properties like those of two-state neurons
- Hopfield
- 1984
(Show Context)
Citation Context ...inuous-time model in symmetric networks. 1 INTRODUCTION For an n-neuron recurrent network, a much-studied and widely-used continuoustime (CT) model is the leaky integrator model (Hertz, et al., 1991; =-=Hopfield, 1984), g-=-iven by a system of nonlinear differential equations: �� i dx i dt = \Gammax i + oe i ( n X j=1 w ij x j + I i ); ts0; i = 1; :::; n; (1) and a related discrete-time (DT) version is the sigmoidal ... |

103 |
Convergent activation dynamics in continuous time networks
- Hirsch
- 1989
(Show Context)
Citation Context ...symmetric (i.e., W is symmetric), the dynamics of both models have been well understood: the CT model (1) is always convergent, namely, every initial state will approach a fixed point asymptotically (=-=Hirsch, 1989-=-; Hertz, et al., 1991; Hopfield, 1984), and the DT model (2) is either convergent or approaches a periodic orbit of period 2 (i.e., a 2-cycle) (Goles, et al., 1985; Marcus & Westervelt, 1989; Koiran, ... |

83 |
Elements of Differentiable Dynamics and Bifurcation Theory
- Ruelle
- 1989
(Show Context)
Citation Context ...envalue of the Jacobian (9) becomes less than \Gamma1 when ff i exceeds the "threshold " c i 0 . During this course of changing ff i , the network undergoes generically a period-doubling bif=-=urcation (Ruelle, 1989-=-), resulting in emergence of some 2-cycles. Thus, in this case the condition (7) in Proposition 3 is also necessary to prevent this type of period-doubling bifurcation from happening around fixed poin... |

41 |
Neural networks and physical systems with collective computational abilities
- Hopfield
- 1982
(Show Context)
Citation Context ...n by a system of nonlinear differential equations: �� i dx i dt = \Gammax i + oe i ( n X j=1 w ij x j + I i ); ts0; i = 1; :::; n; (1) and a related discrete-time (DT) version is the sigmoidal mod=-=el (Hopfield, 1982-=-; Marcus & Westervelt, 1989), specified by a system of nonlinear difference equations: x i (t + 1) = oe i ( n X j=1 w ij x j (t) + I i ); t = 0; 1; :::; i = 1; :::; n; (2) where x i (t), taking values... |

32 |
Neural networks and NP-complete optimization problems: a performance study on the graph bisection problem
- Peterson, JR
- 1988
(Show Context)
Citation Context ...; see (Hertz, et al., 1991, Chapter 4) for an overview. Often, the neuron gains �� i are also modified while the network is evolving. A popular algorithm of this kind uses mean field annealing (MF=-=A) (Peterson & Anderson, 1988-=-) to solve optimization problems, in which small neuron gains are used initially, and increased gradually. Similar situations also happen in some learning algorithms. In practice, a discretized model ... |

17 |
Stability of fixed points and periodic orbits and bifurcations in analog neural networks. Neural Networks
- Blum, Wang
- 1992
(Show Context)
Citation Context ...P ij w ij x j + I i ) = 0). However, as the result of discretization, fixed points may have different asymptotic stability (Wang & Blum, 1992) and periodic points that are not fixed points may occur (=-=Blum & Wang, 1992-=-; Marcus & Westervelt, 1989) in the DT model, especially when all ff i = 1. Nevertheless, the discretized DT model retains the same type of the global parallel dynamics and sequential dynamics of (2),... |

17 | On the convergence properties of the hopfield model
- Bruck
- 1990
(Show Context)
Citation Context ...(i.e., a 2-cycle) (Goles, et al., 1985; Marcus & Westervelt, 1989; Koiran, 1994). For results and analyses of fixed points and cycles in networks that are not necessarily symmetric, see (Brown, 1992; =-=Bruck, 1990-=-; Goles, 1986). For a given symmetric network (n, W , oe i , I i ), the existence of possible 2-cycles in its discrete-time operation is sometimes trouble-some and undesirable, especially in associati... |

17 |
Decreasing energy functions as a tool for studying threshold networks. Discrete Applied Mathematics 12
- Goles, Fogelman-Soulie, et al.
- 1985
(Show Context)
Citation Context ...will approach a fixed point asymptotically (Hirsch, 1989; Hertz, et al., 1991; Hopfield, 1984), and the DT model (2) is either convergent or approaches a periodic orbit of period 2 (i.e., a 2-cycle) (=-=Goles, et al., 1985-=-; Marcus & Westervelt, 1989; Koiran, 1994). For results and analyses of fixed points and cycles in networks that are not necessarily symmetric, see (Brown, 1992; Bruck, 1990; Goles, 1986). For a given... |

16 |
Dynamics of iterated-map neural networks
- Marcus, Westervelt
- 1989
(Show Context)
Citation Context ... nonlinear differential equations: �� i dx i dt = \Gammax i + oe i ( n X j=1 w ij x j + I i ); ts0; i = 1; :::; n; (1) and a related discrete-time (DT) version is the sigmoidal model (Hopfield, 19=-=82; Marcus & Westervelt, 1989-=-), specified by a system of nonlinear difference equations: x i (t + 1) = oe i ( n X j=1 w ij x j (t) + I i ); t = 0; 1; :::; i = 1; :::; n; (2) where x i (t), taking values in a compact interval [a; ... |

15 |
Dynamics of discrete time, continuous state Hopfield networks
- Koiran
- 1994
(Show Context)
Citation Context ...ch, 1989; Hertz, et al., 1991; Hopfield, 1984), and the DT model (2) is either convergent or approaches a periodic orbit of period 2 (i.e., a 2-cycle) (Goles, et al., 1985; Marcus & Westervelt, 1989; =-=Koiran, 1994-=-). For results and analyses of fixed points and cycles in networks that are not necessarily symmetric, see (Brown, 1992; Bruck, 1990; Goles, 1986). For a given symmetric network (n, W , oe i , I i ), ... |

3 |
Discrete-time versus continuous-time neural networks
- Wang, Blum
- 1992
(Show Context)
Citation Context ...n (4)), if and only if it is a fixed point of (1) (i.e., \Gammax i + oe i ( P ij w ij x j + I i ) = 0). However, as the result of discretization, fixed points may have different asymptotic stability (=-=Wang & Blum, 1992-=-) and periodic points that are not fixed points may occur (Blum & Wang, 1992; Marcus & Westervelt, 1989) in the DT model, especially when all ff i = 1. Nevertheless, the discretized DT model retains t... |

2 |
Matrix tests for period 1 and 2 limit cycles in discrete threshold networks
- Brown
- 1992
(Show Context)
Citation Context ... of period 2 (i.e., a 2-cycle) (Goles, et al., 1985; Marcus & Westervelt, 1989; Koiran, 1994). For results and analyses of fixed points and cycles in networks that are not necessarily symmetric, see (=-=Brown, 1992-=-; Bruck, 1990; Goles, 1986). For a given symmetric network (n, W , oe i , I i ), the existence of possible 2-cycles in its discrete-time operation is sometimes trouble-some and undesirable, especially... |

2 | Fixed point behaviour of threshold functions on a finite set - Goles - 1982 |

2 |
Antisymmetrical neural networks
- Goles
- 1986
(Show Context)
Citation Context ...cle) (Goles, et al., 1985; Marcus & Westervelt, 1989; Koiran, 1994). For results and analyses of fixed points and cycles in networks that are not necessarily symmetric, see (Brown, 1992; Bruck, 1990; =-=Goles, 1986-=-). For a given symmetric network (n, W , oe i , I i ), the existence of possible 2-cycles in its discrete-time operation is sometimes trouble-some and undesirable, especially in associative memory and... |