Results 1 - 10
of
13
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
"... Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usabil-ity and perceived quality. Most NLG sys-tems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural vari-ation of human ..."
Abstract
-
Cited by 4 (1 self)
- Add to MetaCart
(Show Context)
Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usabil-ity and perceived quality. Most NLG sys-tems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural vari-ation of human language. They are also not easily scaled to systems covering mul-tiple domains and languages. This pa-per presents a statistical language gener-ator based on a semantically controlled Long Short-term Memory (LSTM) struc-ture. The LSTM generator can learn from unaligned data by jointly optimising sen-tence planning and surface realisation us-ing a simple cross entropy training crite-rion, and language variation can be eas-ily achieved by sampling from output can-didates. With fewer heuristics, an objec-tive evaluation in two differing test do-mains showed the proposed method im-proved performance compared to previ-ous methods. Human judges scored the LSTM system higher on informativeness and naturalness and overall preferred it to the other systems. 1
Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking
- In Proceedings of SIGdial. Association for Computational Linguistics
, 2015
"... The natural language generation (NLG) component of a spoken dialogue system (SDS) usually needs a substantial amount of handcrafting or a well-labeled dataset to be trained on. These limitations add sig-nificantly to development costs and make cross-domain, multi-lingual dialogue sys-tems intractabl ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
(Show Context)
The natural language generation (NLG) component of a spoken dialogue system (SDS) usually needs a substantial amount of handcrafting or a well-labeled dataset to be trained on. These limitations add sig-nificantly to development costs and make cross-domain, multi-lingual dialogue sys-tems intractable. Moreover, human lan-guages are context-aware. The most nat-ural response should be directly learned from data rather than depending on pre-defined syntaxes or rules. This paper presents a statistical language generator based on a joint recurrent and convolu-tional neural network structure which can be trained on dialogue act-utterance pairs without any semantic alignments or pre-defined grammar trees. Objective metrics suggest that this new model outperforms previous methods under the same experi-mental conditions. Results of an evalua-tion by human judges indicate that it pro-duces not only high quality but linguisti-cally varied utterances which are preferred compared to n-gram and rule-based sys-tems. 1
Bilingual continuous-space language model growing for statistical machine translation
- Audio, Speech, and Language Processing, IEEE/ACM Transactions on
, 2015
"... Abstract—Larger-gram language models (LMs) perform better in statistical machine translation (SMT). However, the existing approaches have two main drawbacks for constructing larger LMs: 1) it is not convenient to obtain larger corpora in the same domain as the bilingual parallel corpora in SMT; 2) m ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Abstract—Larger-gram language models (LMs) perform better in statistical machine translation (SMT). However, the existing approaches have two main drawbacks for constructing larger LMs: 1) it is not convenient to obtain larger corpora in the same domain as the bilingual parallel corpora in SMT; 2) most of the previous studies focus on monolingual information from the target corpora only, and redundant-grams have not been fully utilized in SMT. Nowadays, continuous-space language model (CSLM), especially neural network language model (NNLM), has been shown great improvement in the estimation accuracies of the probabilities for predicting the target words. However, most of these CSLM and NNLM approaches still consider monolingual information only or require additional corpus. In this paper, we propose a novel neural network based bilingual LM growing method. Compared to the existing approaches, the proposed method enables us to use bilingual parallel corpus for LM growing in SMT. The results show that our new method outperforms the existing approaches on both SMT performance and computational efficiency significantly. Index Terms—Continuous-space language model, language model growing (LMG), neural network language model, statistical
1 Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
, 2015
"... The natural language generation (NLG) component of a spoken dialogue system (SDS) usu-ally needs a substantial amount of handcrafting or a well-labeled dataset to be trained on. These limitations add significantly to development costs and make cross-domain, multi-lingual dia-logue systems intractabl ..."
Abstract
- Add to MetaCart
(Show Context)
The natural language generation (NLG) component of a spoken dialogue system (SDS) usu-ally needs a substantial amount of handcrafting or a well-labeled dataset to be trained on. These limitations add significantly to development costs and make cross-domain, multi-lingual dia-logue systems intractable. Moreover, human languages are context-aware. The most natural response should be directly learned from data rather than depending on predefined syntaxes or rules. This paper presents a statistical language generator based on a joint recurrent and convolu-tional neural network structure which can be trained on dialogue act-utterance pairs without any semantic alignments or predefined grammar trees. Objective metrics suggest that this new model outperforms previous methods under the same experimental conditions. Results of an evalu-ation by human judges indicate that it produces not only high quality but linguistically varied utterances which are preferred compared to n-gram and rule-based systems. 1
English to Chinese Translation: How Chinese Character Matters?
"... Word segmentation is helpful in Chinese nat-ural language processing in many aspects. However it is showed that different word seg-mentation strategies do not affect the per-formance of Statistical Machine Translation (SMT) from English to Chinese significant-ly. In addition, it will cause some conf ..."
Abstract
- Add to MetaCart
(Show Context)
Word segmentation is helpful in Chinese nat-ural language processing in many aspects. However it is showed that different word seg-mentation strategies do not affect the per-formance of Statistical Machine Translation (SMT) from English to Chinese significant-ly. In addition, it will cause some confu-sions in the evaluation of English to Chinese SMT. So we make an empirical attempt to translation English to Chinese in the charac-ter level, in both the alignment model and lan-guage model. A series of empirical compari-son experiments have been conducted to show how different factors affect the performance of character-level English to Chinese SMT. We also apply the recent popular continuous s-pace language model into English to Chinese SMT. The best performance is obtained with the BLEU score 41.56, which improve base-line system (40.31) by around 1.2 BLEU s-core. Correspondence author. yThank all the reviewers for valuable comments and sug-
1 Deep Learning Background
"... What is Deep Learning? A family of methods that uses deep architectures to learn high-level feature representations (p.2) Example of Trainable Features [Lee et al., 2009] ..."
Abstract
- Add to MetaCart
(Show Context)
What is Deep Learning? A family of methods that uses deep architectures to learn high-level feature representations (p.2) Example of Trainable Features [Lee et al., 2009]
A Comparison between Count and Neural Network Models Based on Joint Translation and Reordering Sequences
"... We propose a conversion of bilingual sentence pairs and the corresponding word alignments into novel linear se-quences. These are joint translation and reordering (JTR) uniquely defined sequences, combining interdepending lexical and alignment dependencies on the word level into a single framework. ..."
Abstract
- Add to MetaCart
(Show Context)
We propose a conversion of bilingual sentence pairs and the corresponding word alignments into novel linear se-quences. These are joint translation and reordering (JTR) uniquely defined sequences, combining interdepending lexical and alignment dependencies on the word level into a single framework. They are constructed in a simple manner while capturing multiple alignments and empty words. JTR sequences can be used to train a variety of models. We investigate the performances of n-gram models with modified Kneser-Ney smoothing, feed-forward and recur-rent neural network architectures when estimated on JTR sequences, and com-pare them to the operation sequence model (Durrani et al., 2013b). Evalua-tions on the IWSLT German→English, WMT German→English and BOLT Chinese→English tasks show that JTR models improve state-of-the-art phrase-based systems by up to 2.2 BLEU.
Recurrent Neural Network based Rule Sequence Model for Statistical Machine Translation
"... The inability to model long-distance depen-dency has been handicapping SMT for years. Specifically, the context independence as-sumption makes it hard to capture the depen-dency between translation rules. In this paper, we introduce a novel recurrent neural network based rule sequence model to incor ..."
Abstract
- Add to MetaCart
(Show Context)
The inability to model long-distance depen-dency has been handicapping SMT for years. Specifically, the context independence as-sumption makes it hard to capture the depen-dency between translation rules. In this paper, we introduce a novel recurrent neural network based rule sequence model to incorporate arbi-trary long contextual information during esti-mating probabilities of rule sequences. More-over, our model frees the translation model from keeping huge and redundant grammars, resulting in more efficient training and de-coding. Experimental results show that our method achieves a 0.9 point BLEU gain over the baseline, and a significant reduction in rule table size for both phrase-based and hierarchi-cal phrase-based systems. 1
Chinese Semantic Role Labeling with Bidirectional Recurrent Neural Networks
"... Traditional approaches to Chinese Seman-tic Role Labeling (SRL) almost heavily re-ly on feature engineering. Even worse, the long-range dependencies in a sentence can hardly be modeled by these method-s. In this paper, we introduce bidirection-al recurrent neural network (RNN) with long-short-term m ..."
Abstract
- Add to MetaCart
Traditional approaches to Chinese Seman-tic Role Labeling (SRL) almost heavily re-ly on feature engineering. Even worse, the long-range dependencies in a sentence can hardly be modeled by these method-s. In this paper, we introduce bidirection-al recurrent neural network (RNN) with long-short-term memory (LSTM) to cap-ture bidirectional and long-range depen-dencies in a sentence with minimal fea-ture engineering. Experimental results on Chinese Proposition Bank (CPB) show a significant improvement over the state-of-the-art methods. Moreover, our model makes it convenient to introduce hetero-geneous resource, which makes a further improvement on our experimental perfor-mance. 1
Graph-Based Collective Lexical Selection for Statistical Machine Translation
"... Lexical selection is of great importance to statistical machine translation. In this paper, we propose a graph-based frame-work for collective lexical selection. The framework is established on a translation graph that captures not only local associ-ations between source-side content words and their ..."
Abstract
- Add to MetaCart
(Show Context)
Lexical selection is of great importance to statistical machine translation. In this paper, we propose a graph-based frame-work for collective lexical selection. The framework is established on a translation graph that captures not only local associ-ations between source-side content words and their target translations but also target-side global dependencies in terms of relat-edness among target items. We also in-troduce a random walk style algorithm to collectively identify translations of source-side content words that are strongly related in translation graph. We validate the ef-fectiveness of our lexical selection frame-work on Chinese-English translation. Ex-periment results with large-scale training data show that our approach significantly improves lexical selection. 1
