Hierarchical probabilistic neural network language model

Hierarchical probabilistic neural network language model (2005)

by Frederic Morin , Yoshua Bengio

Venue:	AISTATS’05
Citations:	11 - 1 self

User correction supplied by mph

Datum	Value	Source
TITLE	Hierarchical probabilistic neural network language model	user correction
AUTHOR NAME	Frederic Morin	user correction
AUTHOR AFFIL	Dept. IRO, Université de Montréal	user correction
AUTHOR ADDR	P.O. Box 6128, Succ. Centre-Ville,; Montreal, H3C 3J7, Qc, Canada	user correction
AUTHOR NAME	Yoshua Bengio	user correction
ABSTRACT	In recent years, variants of a neural network architecture for statistical language modeling have been proposed and successfully applied, e.g. in the language modeling component of speech recognizers. The main advantage of these architectures is that they learn an embedding for words (or other symbols) in a continuous space that helps to smooth the language model and provide good generalization even when the number of training examples is insufficient. However, these models are extremely slow in comparison to the more commonly used n-gram models, both for training and recognition. As an alternative to an importance sampling method proposed to speed-up training, we introduce a hierarchical decomposition of the conditional probabilities that yields a speed-up of about 200 both during training and recognition. The hierarchical decomposition is a binary hierarchical clustering constrained by the prior knowledge extracted from the WordNet semantic hierarchy.	user correction
YEAR	2005	INFERENCE
VENUE	AISTATS’05	user correction
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	246--252	INFERENCE
CITATIONS	26 found	ParsCit 1.0