DMCA
Sequencediscriminative training of deep neural networks (2013)
| Venue: | in Proc. INTERSPEECH |
| Citations: | 59 - 4 self |
BibTeX
@INPROCEEDINGS{Ghoshal13sequencediscriminativetraining,
author = {Arnab Ghoshal and Daniel Povey},
title = {Sequencediscriminative training of deep neural networks},
booktitle = {in Proc. INTERSPEECH},
year = {2013}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Sequence-discriminative training of deep neural networks (DNNs) is investigated on a 300 hour American English conversational telephone speech task. Different sequencediscriminative criteria — maximum mutual information (MMI), minimum phone error (MPE), state-level minimum Bayes risk (sMBR), and boosted MMI — are compared. Two different heuristics are investigated to improve the performance of the DNNs trained using sequence-based criteria — lattices are regenerated after the first iteration of training; and, for MMI and BMMI, the frames where the numerator and denominator hypotheses are disjoint are removed from the gradient computation. Starting from a competitive DNN baseline trained using cross-entropy, different sequence-discriminative criteria are shown to lower word error rates by 8-9 % relative, on average. Little difference is noticed between the different sequencebased criteria that are investigated. The experiments are done using the open-source Kaldi toolkit, which makes it possible for the wider community to reproduce these results. Index Terms: speech recognition, deep learning, sequencecriterion training, neural networks, reproducible research
Keyphrases
deep neural network sequencediscriminative training speech recognition little difference competitive dnn baseline different sequence-discriminative criterion wider community neural network 8-9 relative denominator hypothesis deep learning minimum phone error state-level minimum bayes risk open-source kaldi toolkit index term sequencecriterion training gradient computation first iteration reproducible research different heuristic sequence-based criterion lattice sequence-discriminative training word error rate
