DMCA
Accelerated parallelizable neural network learning algorithm for speech recognition (2011)
| Venue: | in Proc. Interspeech |
| Citations: | 6 - 5 self |
BibTeX
@INPROCEEDINGS{Yu11acceleratedparallelizable,
author = {Dong Yu and Li Deng},
title = {Accelerated parallelizable neural network learning algorithm for speech recognition},
booktitle = {in Proc. Interspeech},
year = {2011}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We describe a set of novel, batch-mode algorithms we developed recently as one key component in scalable, deep neural network based speech recognition. The essence of these algorithms is to structure the singlehidden-layer neural network so that the upper-layer’s weights can be written as a deterministic function of the lower-layer’s weights. This structure is effectively exploited during training by plugging in the deterministic function to the least square error objective function while calculating the gradients. Accelerating techniques are further exploited to make the weight updates move along the most promising directions. The experiments on TIMIT frame-level phone and phonestate classification show strong results. In particular, the error rate is strictly monotonically dropping as the minibatch size increases. This demonstrates the potential for the proposed batch-mode algorithms in large scale speech recognition since they are easily parallelizable across computers. Index Terms: neural network, scalability, structure, constraints, FISTA acceleration, optimization, pseudoinverse, weighted LSE, phone state classification, speech recognition, deep learning 1.
Keyphrases
speech recognition accelerated parallelizable neural network deterministic function batch-mode algorithm index term promising direction large scale speech recognition deep learning singlehidden-layer neural network fista acceleration phonestate classification show strong result upper-layer weight neural network square error objective function key component phone state classification timit frame-level phone lower-layer weight error rate minibatch size increase deep neural network
