Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition

DMCA

Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition (2012)

Cached

Download Links

by George E. Dahl , Dong Yu , Li Deng , Alex Acero

Venue:	IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING
Citations:	222 - 41 self

BibTeX

@INPROCEEDINGS{Dahl12context-dependentpre-trained,
    author = {George E. Dahl and Dong Yu and Li Deng and Alex Acero},
    title = {Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition},
    booktitle = {IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING},
    year = {2012},
    publisher = {}
}

OpenURL

Abstract

We propose a novel context-dependent (CD) model for large vocabulary speech recognition (LVSR) that leverages recent advances in using deep belief networks for phone recognition. We describe a pretrained deep neural network hidden Markov model (DNN-HMM) hybrid architecture that trains the DNN to produce a distribution over senones (tied triphone states) as its output. The deep belief network pre-training algorithm is a robust and often helpful way to initialize deep neural networks generatively that can aid in optimization and reduce generalization error. We illustrate the key components of our model, describe the procedure for applying CD-DNN-HMMs to LVSR, and analyze the effects of various modeling choices on performance. Experiments on a challenging business search dataset demonstrate that CD-DNN-HMMs can significantly outperform the conventional context-dependent Gaussian mixture model (GMM)-HMMs, with an absolute sentence accuracy improvement of 5.8 % and 9.2 % (or relative error reduction of 16.0 % and 23.2%) over the CD-GMM-HMMs trained using the minimum phone error rate (MPE) and maximum likelihood (ML) criteria, respectively.

Keyphrases

large vocabulary speech recognition context-dependent pre-trained deep neural network deep belief network deep neural network index term speech recognition context-dependent phone deep belief network pre-training algorithm generalization error conventional context-dependent gaussian mixture model absolute sentence accuracy improvement challenging business search dataset demonstrate triphone state minimum phone error rate novel context-dependent hybrid architecture recent advance phone recognition helpful way maximum likelihood error reduction key component

DMCA