DMCA
Modeling systems with internal state using Evolino (2005)
Cached
Download Links
| Venue: | In Proc. of the 2005 conference on genetic and evolutionary computation (GECCO |
| Citations: | 6 - 1 self |
BibTeX
@INPROCEEDINGS{Wierstra05modelingsystems,
author = {Daan Wierstra and Faustino J. Gomez and Jürgen Schmidhuber},
title = {Modeling systems with internal state using Evolino},
booktitle = {In Proc. of the 2005 conference on genetic and evolutionary computation (GECCO},
year = {2005},
pages = {1795--1802},
publisher = {ACM Press}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Existing Recurrent Neural Networks (RNNs) are limited in their ability to model dynamical systems with nonlinearities and hidden internal states. Here we use our general framework for sequence learning, EVOlution of recurrent systems with LINear Outputs (Evolino), to discover good RNN hidden node weights through evolution, while using linear regression to compute an optimal linear mapping from hidden state to output. Using the Long Short-Term Memory RNN Architecture, Evolino outperforms previous state-of-the-art methods on several tasks: 1) context-sensitive languages, 2) multiple superimposed sine waves. Categories and Subject Descriptors I.2.6 [Artificial Intelligence]: Learning—Connectionism and neural nets
Keyphrases
internal state optimal linear mapping previous state-of-the-art method good rnn hidden node weight learning connectionism general framework linear output dynamical system neural net several task long short-term memory rnn architecture subject descriptor context-sensitive language sine wave sequence learning hidden state recurrent system linear regression recurrent neural network artificial intelligence hidden internal state
