Results 1 
7 of
7
Fusion of knowledgebased and datadriven approaches to grammar induction
"... Using different sources of information for grammar induction results in grammars that vary in coverage and precision. Fusing such grammars with a strategy that exploits their strengths while minimizing their weaknesses is expected to produce grammars with superior performance. We focus on the fusion ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Using different sources of information for grammar induction results in grammars that vary in coverage and precision. Fusing such grammars with a strategy that exploits their strengths while minimizing their weaknesses is expected to produce grammars with superior performance. We focus on the fusion of grammars produced using a knowledgebased approach using lexicalized ontologies and a datadriven approach using semantic similarity clustering. We propose various algorithms for finding the mapping between the (nonterminal) rules generated by each grammar induction algorithm, followed by rule fusion. Three fusion approaches are investigated: early, mid and late fusion. Results show that late fusion provides the best relative Fmeasure performance improvement by 20%. Index Terms: spoken dialogue systems, corpusbased grammar induction, ontologybased grammar induction, grammar fusion
Lexical Decoding Based on the Combination of CategoryBased Stochastic Models and WordCategory Distribution Models
, 2001
"... Lexical decoding is the obtaining of the most probable sequence of categories associated to a sequence of words. This paper describes two lexical decoding combined models which are based on a stochastic categorybased model and a probabilistic model of word distribution into linguistic categories ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Lexical decoding is the obtaining of the most probable sequence of categories associated to a sequence of words. This paper describes two lexical decoding combined models which are based on a stochastic categorybased model and a probabilistic model of word distribution into linguistic categories. In the rst combined model, the stochastic categorybased model is a Stochastic ContextFree Grammar, and in the second combined model, the stochastic categorybased model is a ngram model. The estimation processes of the models are described in detail. Finally, experiments on the Wall Street Journal corpus are reported.
Using Perfect Sampling in Parameter Estimation of a Whole Sentence Maximum Entropy Language Model
, 2000
"... The Maximum Entropy principle (ME) is an ap propriate framework for combining information of a diverse nature from several sources into the same language model. In order to incorporate longdistance information into the ME framework in a language model, a Whole Sentence Maximum Entropy Language Mod ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
The Maximum Entropy principle (ME) is an ap propriate framework for combining information of a diverse nature from several sources into the same language model. In order to incorporate longdistance information into the ME framework in a language model, a Whole Sentence Maximum Entropy Language Model (WSME) could be used. Until now MonteCarlo Markov Chains (MCMC) sampling techniques has been used to estimate the paramenters of the WSME model. In this paper, we propose the application of another sampling technique: the Perfect Sampling (PS). The experiment has shown a reduction of 30% in the perplexity of the WSME model over the trigram model and a reduc tion of 2% over the WSME model trained with MCMC.
A Hybrid Language Model based on Stochastic Contextfree Grammars ⋆
"... Abstract. This paper explores the use of initial Stochastic ContextFree Grammars (SCFG) obtained from a treebank corpus for the learning of SCFG by means of estimation algorithms. A hybrid language model is defined as a combination of a wordbased ngram, which is used to capture the local relation ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. This paper explores the use of initial Stochastic ContextFree Grammars (SCFG) obtained from a treebank corpus for the learning of SCFG by means of estimation algorithms. A hybrid language model is defined as a combination of a wordbased ngram, which is used to capture the local relations between words, and a categorybased SCFG with a word distribution into categories, which is defined to represent the longterm relations between these categories. Experiments on the UPenn Treebank corpus are reported. These experiments have been carried out in terms of the test set perplexity and the word error rate in a speech recognition experiment. 1
unknown title
"... application. Furthermore, other structure knowledge such as context free grammar and link grammar [10, 11] is introduced to language models for improving their performances. To combine statistical information from multiple sources, maximum entropy (ME) LM is presented [12]. Unlike the linear interpo ..."
Abstract
 Add to MetaCart
application. Furthermore, other structure knowledge such as context free grammar and link grammar [10, 11] is introduced to language models for improving their performances. To combine statistical information from multiple sources, maximum entropy (ME) LM is presented [12]. Unlike the linear interpolation and backoff model, in which separate models must be constructed, ME only build a combined model on which each information source imposes a set of constraints. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the ME solution. ME takes all the previous words as possible features, so training ME model is computational challenging and sometimes almost infeasible. From the review above, we know that the improvement in language models mainly focuses on three aspects: the solution of longdistance dependencies, the integration of linguistic knowledge into LM, and the general framework that combines all kinds of knowledge. In Chinese, shallow parsing has gotten some promising results [13, 14]. However, due to the lacks of the fineannotated corpus (such as Treebanks) and competitive syntactic parser, it is infeasible to build a language model that depends on the complete parsing technique such as Chelba [9]. In this paper, an improved language model incorporating linguistic structure into maximum entropy framework is presented. The proposed model combines trigram and structure knowledge of base phrase in which trigram is used to capture the local relation between words, while structure knowledge
Combining patternbased CRFs and weighted contextfree grammars
"... Abstract. We consider two models for the sequence labeling (tagging) problem. The first one is a PatternBased Conditional Random Field (PB), in which the energy of a string (chain labeling) x = x1... xn ∈ Dn is a sum of terms over intervals [i, j] where each term is nonzero only if the substring x ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. We consider two models for the sequence labeling (tagging) problem. The first one is a PatternBased Conditional Random Field (PB), in which the energy of a string (chain labeling) x = x1... xn ∈ Dn is a sum of terms over intervals [i, j] where each term is nonzero only if the substring xi... xj equals a prespecified word w ∈ Λ. The second model is a Weighted ContextFree Grammar (WCFG) frequently used for natural language processing. PB and WCFG encode local and nonlocal interactions respectively, and thus can be viewed as complementary. We propose a Grammatical PatternBased CRF model (GPB) that combines the two in a natural way. We argue that it has certain advantages over existing approaches such as the Hybrid model of [3] that combines Ngrams and WCFGs. The focus of this paper is to analyze the complexity of inference tasks in a GPB such as computing MAP. We present a polynomialtime algorithm for general GPBs and a faster version for a special case that we call Interaction Grammars. Key words: sequence tagging, patternbased CRFs, weighted contextfree grammars 1