Results 1  10
of
60
Conditional random fields: Probabilistic models for segmenting and labeling sequence data
, 2001
"... We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions ..."
Abstract

Cited by 3162 (84 self)
 Add to MetaCart
(Show Context)
We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and naturallanguage data. 1.
A Stochastic FiniteState WordSegmentation Algorithm For Chinese
 Computational Linguistics
, 1996
"... Chinese text into dictionary entries and productively derived words, and providing pronunciations for these words; the method incorporates a classbased model in its treatment of personal names. We also evaluate the system's performance, taking into account the fact that people often do not agr ..."
Abstract

Cited by 155 (9 self)
 Add to MetaCart
Chinese text into dictionary entries and productively derived words, and providing pronunciations for these words; the method incorporates a classbased model in its treatment of personal names. We also evaluate the system's performance, taking into account the fact that people often do not agree on a single seg mentation.
The Design Principles of a Weighted FiniteState Transducer Library
, 2002
"... We describe the algorithmic and software design principles of an objectoriented library for weighted finitestate transducers. By taking advantage of the theory of rational power series, we were able to achieve high degrees of generality, modularity and irredundancy, while attaining competitive eff ..."
Abstract

Cited by 106 (20 self)
 Add to MetaCart
(Show Context)
We describe the algorithmic and software design principles of an objectoriented library for weighted finitestate transducers. By taking advantage of the theory of rational power series, we were able to achieve high degrees of generality, modularity and irredundancy, while attaining competitive efficiency in demanding speech processing applications involving weighted automata of more than 10 7 states and transitions. Besides its mathematical foundation, the design also draws from important ideas in algorithm design and programming languages: dynamic programming and shortestpaths algorithms over general semirings, objectoriented programming, lazy evaluation and memoization.
OpenFst: A general and efficient weighted finitestate transducer library. Implementation and Application of Automata
, 2007
"... Abstract. We describe OpenFst, an opensource library for weighted finitestate transducers (WFSTs). OpenFst consists of a C++ template library with efficient WFST representations and over twentyfive operations for constructing, combining, optimizing, and searching them. At the shellcommand level, ..."
Abstract

Cited by 98 (12 self)
 Add to MetaCart
(Show Context)
Abstract. We describe OpenFst, an opensource library for weighted finitestate transducers (WFSTs). OpenFst consists of a C++ template library with efficient WFST representations and over twentyfive operations for constructing, combining, optimizing, and searching them. At the shellcommand level, there are corresponding transducer file representations and programs that operate on them. OpenFst is designed to be both very efficient in time and space and to scale to very large problems. This library has key applications speech, image, and natural language processing, pattern and string matching, and machine learning. We give an overview of the library, examples of its use, details of its design that allow customizing the labels, states, and weights and the lazy evaluation of many of its operations. Further information and a download of the OpenFst library can be obtained from
SEMIRING FRAMEWORKS AND ALGORITHMS FOR SHORTESTDISTANCE PROBLEMS
, 2002
"... We define general algebraic frameworks for shortestdistance problems based on the structure of semirings. We give a generic algorithm for finding singlesource shortest distances in a weighted directed graph when the weights satisfy the conditions of our general semiring framework. The same algorit ..."
Abstract

Cited by 86 (20 self)
 Add to MetaCart
We define general algebraic frameworks for shortestdistance problems based on the structure of semirings. We give a generic algorithm for finding singlesource shortest distances in a weighted directed graph when the weights satisfy the conditions of our general semiring framework. The same algorithm can be used to solve efficiently classical shortest paths problems or to find the kshortest distances in a directed graph. It can be used to solve singlesource shortestdistance problems in weighted directed acyclic graphs over any semiring. We examine several semirings and describe some specific instances of our generic algorithms to illustrate their use and compare them with existing methods and algorithms. The proof of the soundness of all algorithms is given in detail, including their pseudocode and a full analysis of their running time complexity.
Statistical language model adaptation: review and perspectives
 Speech Communication
, 2004
"... Speech recognition performance is severely affected when the lexical, syntactic, or semantic characteristics of the discourse in the training and recognition tasks differ. The aim of language model adaptation is to exploit specific, albeit limited, knowledge about the recognition task to compensate ..."
Abstract

Cited by 70 (0 self)
 Add to MetaCart
(Show Context)
Speech recognition performance is severely affected when the lexical, syntactic, or semantic characteristics of the discourse in the training and recognition tasks differ. The aim of language model adaptation is to exploit specific, albeit limited, knowledge about the recognition task to compensate for this mismatch. More generally, an adaptive language model seeks to maintain an adequate representation of the current task domain under changing conditions involving potential variations in vocabulary, syntax, content, and style. This paper presents an overview of the major approaches proposed to address this issue, and offers some perspectives regarding their comparative merits and associated tradeoffs. Ó 2003 Elsevier B.V. All rights reserved. 1.
Finite State Transducers with Predicates and Identities
 Grammars
, 2001
"... An extension to finite state transducers is presented, in which atomic symbols are replaced by arbitrary predicates over symbols. The extension is motivated by applications in natural language processing (but may be more widely applicable) as well as by the observation that transducers with predicat ..."
Abstract

Cited by 29 (0 self)
 Add to MetaCart
An extension to finite state transducers is presented, in which atomic symbols are replaced by arbitrary predicates over symbols. The extension is motivated by applications in natural language processing (but may be more widely applicable) as well as by the observation that transducers with predicates generally have fewer states and fewer transitions. Although the extension is fairly trivial for finite state acceptors, the introduction of predicates is more interesting for transducers. It is shown how various operations on transducers (e.g. composition) can be implemented, as well as how the transducer determinization algorithm can be generalized for predicateaugmented finite state transducers.
Query segmentation for web search
"... This paper describes a query segmentation method for search engines supporting inverse lookup of words and phrases. Data mining in query logs and document corpora is used to produce segment candidates and compute connexity measures. Candidates are considered in context of the whole query, and a l ..."
Abstract

Cited by 23 (0 self)
 Add to MetaCart
(Show Context)
This paper describes a query segmentation method for search engines supporting inverse lookup of words and phrases. Data mining in query logs and document corpora is used to produce segment candidates and compute connexity measures. Candidates are considered in context of the whole query, and a list of the most likely segmentations is generated, with each segment attributed with a connexity value. For each segmentation a segmentation score is computed from connexity values of nontrivial segments, which can be used as a sorting criterion for the segmentations. We also point to a relevancy improvement in query evaluation model by means of proximity penalty. Keywords web search, query processing, data mining, query segmentation, query evaluation 1.
Network Optimizations for Large Vocabulary Speech Recognition
 Speech Communication
, 1998
"... The redundancy and the size of networks in largevocabulary speech recognition systems can have a critical effect on their overall performance. We describe the use of two new algorithms: weighted determinization and minimization [12]. These algorithms transform recognition labeled networks into equi ..."
Abstract

Cited by 22 (6 self)
 Add to MetaCart
The redundancy and the size of networks in largevocabulary speech recognition systems can have a critical effect on their overall performance. We describe the use of two new algorithms: weighted determinization and minimization [12]. These algorithms transform recognition labeled networks into equivalent ones that require much less time and space in largevocabulary speech recognition. They are both optimal: weighted determinization eliminates the number of alternatives at each state to the minimum, and weighted minimization reduces the size of deterministic networks to the smallest possible number of states and transitions. These algorithms generalize classical automata determinization and minimization to deal properly with the probabilities of alternative hypotheses and with the relationships between units (distributions, phones, words) at different levels in the recognition system. We illustrate their use in several applications, and report the results of our experiments. Key words...