## The Computational Power of Discrete Hopfield Nets with Hidden Units (1996)

Venue: | Neural Computation |

Citations: | 11 - 6 self |

### BibTeX

@ARTICLE{Orponen96thecomputational,

author = {Pekka Orponen},

title = {The Computational Power of Discrete Hopfield Nets with Hidden Units},

journal = {Neural Computation},

year = {1996},

volume = {8},

pages = {403--415}

}

### OpenURL

### Abstract

We prove that polynomial size discrete Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial space-bounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks with polynomially bounded interconnection weights compute exactly the class of functions P/poly, i.e., the class computed by polynomial time-bounded nonuniform Turing machines.

### Citations

1780 |
Neural networks and physical systems with emergent collective computation abilities
- Hopfield
- 1982
(Show Context)
Citation Context ...s compute exactly the class of functions P/poly, i.e., the class computed by polynomial time-bounded nonuniform Turing machines. 1 Introduction We investigate the power of discrete Hopfield networks (=-=Hopfield, 1982-=-) as general computational devices. Our main interest is in the problem of Boolean function computation by symmetric networks of weighted threshold logic units; but for the constructions, we also need... |

846 | A logical calculus of ideas immanent in nervous activity - Mcculloch, Pitts - 1943 |

638 |
1984] “Neurons with graded response have collective properties like those of two-state neurons
- Hopfield
(Show Context)
Citation Context ...uted by such networks would nevertheless be of some interest in the associative memory context. Of more general interest would be the study of the continuous-time version of Hopfield's network model (=-=Hopfield 1984-=-; Hopfield and Tank 1985). It will be an exciting broad research task to define the appropriate notions of computability and complexity in this model, and attempt to characterize its computational pow... |

543 |
Neural Computation of Decisions in Optimization Problems
- Hopfield, Tank
- 1985
(Show Context)
Citation Context ...tworks would nevertheless be of some interest in the associative memory context. Of more general interest would be the study of the continuous-time version of Hopfield's network model (Hopfield 1984; =-=Hopfield and Tank 1985-=-). It will be an exciting broad research task to define the appropriate notions of computability and complexity in this model, and attempt to characterize its computational power. Acknowledgment I wis... |

489 |
Computation: Finite and Infinite Machines
- Minsky
- 1967
(Show Context)
Citation Context ...d Goodman 1988), or indeed even on a network where the update order is a priori totally undetermined (Orponen 1995). Following the early work of McCulloch and Pitts (1943) and Kleene (1956) (see also =-=Minsky 1972-=-), it has been customary to think of finite (asymmetric) networks of threshold logic units as equivalent to finite automata (for recent work along these lines see, e.g., Alon et al. 1991; Horne and Hu... |

394 | Representation of events in nerve nets and finite automata - Kleene - 1956 |

181 |
Structural Complexity
- Balcázar, Dı́az, et al.
- 1990
(Show Context)
Citation Context .../poly of functions computed by polynomial size Boolean circuits or, equivalently, polynomial time Turing machines with a polynomially bounded number of nonuniform "advice bits" (Karp and Lip=-=ton 1982; Balc'azar et al. 1988-=-) 1 . On the other hand, if computation times are not bounded, then a relatively straightforward argument shows that the class of functions computed by polynomial size asymmetric nets equals the class... |

89 | Analog computation via neural networks - Siegelmann, Sontag - 1994 |

83 |
Simple local search problems that are hard to solve
- Schaffer, Yannakakis
- 1991
(Show Context)
Citation Context ... demonstrated by Haken (1989). The existence of networks with exponentially long asynchronous transients is now known to follow also from the general theory of local search for optimization problems (=-=Schaffer and Yannakakis 1991-=-). In this paper, we prove that despite their constrained dynamics, computationally symmetric networks lose nothing of their power: specifically, symmetric polynomial size networks with unbounded weig... |

55 |
Circuit Complexity and Neural Networks
- Parberry
- 1994
(Show Context)
Citation Context ... al. 1988, p. 111). This includes circuits using threshold logic gates, as any threshold function on k variables can be implemented as an AND/OR/NOT-circuit of size O(k 2 log 2 k) and depth O(log k) (=-=Parberry 1994-=-, p. 173). 3 Simulating Turing Machines with Asymmetric Nets Simulating space-bounded Turing machines with asymmetric neural nets is fairly straightforward. Theorem 3.1 PNETS = PSPACE/poly. fl fi ffl ... |

51 |
Turing machines that take advice. L’Enseignement Mathématique
- Karp, Lipton
- 1982
(Show Context)
Citation Context ...cides with the class P/poly of functions computed by polynomial size Boolean circuits or, equivalently, polynomial time Turing machines with a polynomially bounded number of nonuniform "advice bi=-=ts" (Karp and Lipton 1982-=-; Balc'azar et al. 1988) 1 . On the other hand, if computation times are not bounded, then a relatively straightforward argument shows that the class of functions computed by polynomial size asymmetri... |

38 |
Efficient simulation of finite automata by neural nets, .f
- Alon, Dewdney, et al.
- 1991
(Show Context)
Citation Context ...e (1956) (see also Minsky 1972), it has been customary to think of finite (asymmetric) networks of threshold logic units as equivalent to finite automata (for recent work along these lines see, e.g., =-=Alon et al. 1991-=-; Horne and Hush 1994; Indyk 1995). However, in Kleene's construction for the equivalence, the input to a net is given as a sequence of pulses, whereas from many of the current applications' point of ... |

36 |
Bounds on the complexity of recurrent neural network implementations of finite state machines
- Horne, Hush
- 1996
(Show Context)
Citation Context ... Minsky 1972), it has been customary to think of finite (asymmetric) networks of threshold logic units as equivalent to finite automata (for recent work along these lines see, e.g., Alon et al. 1991; =-=Horne and Hush 1994-=-; Indyk 1995). However, in Kleene's construction for the equivalence, the input to a net is given as a sequence of pulses, whereas from many of the current applications' point of view it would be more... |

33 | Recursive neural networks for associative memory - Kamp, Hasler - 1990 |

28 | A primer on the complexity theory of neural networks - Parberry - 1990 |

27 |
Neural and Automata Networks
- Goles, Mart
- 1990
(Show Context)
Citation Context ... a sequence of symmetric edges and their interconnecting units, whose behavior is coordinated by a system clock (Figures 2, 3); and a binary counter network due to Goles and Mart'inez (1989; see also =-=Goles and Mart'inez 1990-=-, pp. 88--95) that can count up to 2 n using about 3n units and O(n 2 ) symmetric edges (Figure 4). An important observation here is that any convergent computation by a network of n units has to term... |

26 | A comparison of the computational power of neural network models - HARTLEY, SZU - 1987 |

24 |
Optimal simulation of automata by neural nets
- Indyk
- 1995
(Show Context)
Citation Context ... been customary to think of finite (asymmetric) networks of threshold logic units as equivalent to finite automata (for recent work along these lines see, e.g., Alon et al. 1991; Horne and Hush 1994; =-=Indyk 1995-=-). However, in Kleene's construction for the equivalence, the input to a net is given as a sequence of pulses, whereas from many of the current applications' point of view it would be more natural to ... |

23 |
A generalized convergence theorem for neural networks
- Bruck, Goodman
- 1988
(Show Context)
Citation Context ...arallel). However, it is known that any computation on a synchronous network can be simulated on an asynchronous network where the updates are performed in a specific sequential order (Tchuente 1986; =-=Bruck and Goodman 1988-=-), or indeed even on a network where the update order is a priori totally undetermined (Orponen 1995). Following the early work of McCulloch and Pitts (1943) and Kleene (1956) (see also Minsky 1972), ... |

23 | Steepest descent can take exponential time for symmetric connection networks - Haken, Luby - 1988 |

21 |
Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating”,Complex Systems
- Goles, Mart́ınez
(Show Context)
Citation Context ...tion of a (3n \Gamma 4)-unit symmetric network with a convergence time of more than 2 n (actually, 2 n + 2 n\Gamma1 \Gamma 3) synchronous update steps. (For the full details of the construction, see (=-=Goles and Mart'inez 1989-=-).) The idea here is that the n units in the upper row implement a binary counter, counting from all 0's to all 1's (in the figure, the unit corresponding to the least significant bit is to the right)... |

21 | On periodical behaviour in societies with symmetric influences - Poljak, Sura - 1983 |

19 | Computing with truly asynchronous threshold logic networks
- Orponen
- 1997
(Show Context)
Citation Context ...us network where the updates are performed in a specific sequential order (Tchuente 1986; Bruck and Goodman 1988), or indeed even on a network where the update order is a priori totally undetermined (=-=Orponen 1995-=-). Following the early work of McCulloch and Pitts (1943) and Kleene (1956) (see also Minsky 1972), it has been customary to think of finite (asymmetric) networks of threshold logic units as equivalen... |

18 | Complexity Issues in Discrete Hopfield Networks”, NeuroCOLT - Floreen, Orponen - 1994 |

18 | G.: Computational power for networks of threshold devices - Lepley, Miller |

17 |
Decreasing energy functions as a tool for studying threshold networks. Discrete Applied Mathematics 12
- Goles, Fogelman-Soulie, et al.
- 1985
(Show Context)
Citation Context ...ost a total of 3 X i;j jw ij j = O(p 2 \Delta max i;j jw ij j) unit state changes. Under synchronous updates a similar bound holds also for nets with negative self-connections (Poljak and Ssura 1983; =-=Goles et al. 1985-=-; Bruck and Goodman 1988), but in this model the network may also converge to oscillate between two alternating states instead of a unique stable state. Thus, in particular, symmetric networks with po... |

17 | Connectionist networks that need exponential time to stabilize. Unpublished manuscript - Haken - 1989 |

15 | Transient length in sequential iterations of threshold functions - Floreen, Goles, et al. - 1983 |

13 | The convergence of symmetric threshold automata - Goles, Olivos - 1981 |

10 | Automata Networks in Computer Science: Theory and Applications - Souli'e, Robert, et al. - 1987 |

9 |
Sequential simulation of parallel iterations and applications. Theoretical Computer Science 48
- Tchuente
- 1986
(Show Context)
Citation Context ...ltaneously in parallel). However, it is known that any computation on a synchronous network can be simulated on an asynchronous network where the updates are performed in a specific sequential order (=-=Tchuente 1986-=-; Bruck and Goodman 1988), or indeed even on a network where the update order is a priori totally undetermined (Orponen 1995). Following the early work of McCulloch and Pitts (1943) and Kleene (1956) ... |

5 | Fixed point behavior of threshold functions on a finite set - Goles - 1982 |

4 |
On characterizations of the class PSPACE/poly. Theoretical Computer Science 52:251--267
- Balc'azar, D'iaz, et al.
- 1987
(Show Context)
Citation Context ...ction mapping pairs of binary strings to binary strings (see, e.g., Balc'azar et al. (1988, p. 7)). A language A ` f0; 1g belongs to the nonuniform complexity class PSPACE/poly (Karp and Lipton 1982; =-=Balc'azar et al. 1987; Balc'aza-=-r et al. 1988, p. 100), if there exist a polynomial space bounded Turing machine M and an "advice" function f : N ! f0; 1g , such that for some polynomial q and all n 2 N , jf(n)jsq(n), and ... |