Learning the Initial State of a Second-Order Recurrent Neural Network During Regular-Language Inference (1994)
| Citations: | 21 - 9 self |
BibTeX
@MISC{Forcada94learningthe,
author = {Mikel L. Forcada and Rafael C. Carrasco},
title = {Learning the Initial State of a Second-Order Recurrent Neural Network During Regular-Language Inference},
year = {1994}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Recent work has shown that second-order recurrent neural networks (2ORNNs) may be used to infer regular languages. This letter presents a modified version of the real-time recurrent learning (RTRL) algorithm used to train 2ORNNs, that learns the initial state in addition to the weights. The results of this modification, which adds extra flexibility at a negligible cost in time complexity, suggest that it may be used to improve the learning of regular languages when the size of the network is small. To appear in Neural Computation, vol. 7 (1995). Introduction: A number of recent papers (Giles et al. 1992a,b; Siegelmann, Sontag and Giles, 1992; Watrous and Kuhn 1992a,b) have explored the ability of second-order recurrent neural networks (2ORNNs) to learn simple regular grammars (that is, with languages accepted by small automata) from positive and negative training word samples. This letter presents a modified version of the real-time recurrent learning (RTRL) algorithm used by these aut...
Citations
| 368 | A Learning Algorithm for Continually Running Fully Recurrent Neural Networks’, Neural computation - Williams, Zipser - 1989 |
| 37 | Extracting and learning an unknown grammar with recurrent neural networks - Giles, Miller, et al. - 1992 |
| 36 | Dynamic construction of finite-state automata from examples using hill-climbing - Tomita - 1982 |
| 20 | Induction of finite--state automata using second-- order recurrent networks - Watrous, Kuhn - 1992 |
| 12 | The complexity of language recognition by neural networks - Siegelmann, Sontag, et al. - 1992 |
