## A Rational Design for a Weighted Finite-State Transducer Library (1998)

Venue: | LECTURE NOTES IN COMPUTER SCIENCE |

Citations: | 66 - 1 self |

### BibTeX

@INPROCEEDINGS{Mohri98arational,

author = {Mehryar Mohri and Fernando Pereira and Michael Riley},

title = {A Rational Design for a Weighted Finite-State Transducer Library},

booktitle = {LECTURE NOTES IN COMPUTER SCIENCE},

year = {1998},

pages = {144--158},

publisher = {Springer}

}

### Years of Citing Articles

### OpenURL

### Abstract

### Citations

8980 |
Introduction to Algorithmics
- Cormen, Leiserson, et al.
- 2001
(Show Context)
Citation Context ...for solving all pairs shortest-paths problems --- closed semirings --- is compatible with the abstract notion of weights we use, we were able to include an efficient version of that generic algorithm =-=[1, 4]-=- in our library. Using the same algorithm and code, we can provide the all-pairs shortest distances when weights are real numbers representing, for example, probabilities, but also when they are strin... |

2541 |
The Design and Analysis of Computer Algorithms
- Aho, Hopcroft, et al.
- 1974
(Show Context)
Citation Context ...a arising in large-vocabulary recognition require careful implementation techniques even for standard algorithms. For example, iterating over the symbol set in the standard DFA minimization algorithm =-=[1]-=- is impractical for sparse DFAs if the symbol set is large, as is the case in language models for large-vocabulary recognition. 1 Weighted acceptors and transducers have also been used in image proces... |

346 | Regular models of phonological rule systems
- Kaplan, Kay
- 1994
(Show Context)
Citation Context ...ammar optimization techniques based on it. In some other areas of language processing, especially dictionaries, morphology and local parsing, finite-state techniques have been used with great success =-=[7, 13]-=-. Complex finite-state mappings between strings can be represented with regular expressions over elementary transductions (input-output mappings) together with relational composition of transductions.... |

317 | Finite-state transducers in language and speech processing
- Mohri
- 1997
(Show Context)
Citation Context ...ansducer composition [17], and acceptor intersection, as well as taking the difference between a weighted acceptor and an unweighted DFA; Equivalence transformations: ffl-elimination, determinization =-=[14, 15]-=- and minimization for unweighted (both the general case [1] and the more efficient acyclic case [20]) and weighted acceptors and transducers [12, 15], removal of inaccessible states and transitions; S... |

140 | A Stochastic Finite-state Word Segmentation Algorithm for Chinese. Computational Linguistics
- Sproat, Gale, et al.
- 1996
(Show Context)
Citation Context ...available in more "ad hoc" speech recognition frameworks. The generality of the library has made it also valuable in other language-processing applications, such as word segmentation for Chi=-=nese text [25]-=-. 1.1 Design Rationale Current speech-recognition systems rely on a variety of probabilistic finite-state models, for instance n-gram language models [21], multiple-pronunciation dictionaries [11], an... |

117 |
On the definition of a family of automata
- Schützenberger
- 1961
(Show Context)
Citation Context ..., and thus with finite automata with weighted transitions. 1 Weighted languages are just formal power series over appropriate semirings, and weighted transductions can be interpreted in a similar way =-=[23, 6, 9, 3]-=-. We were thus led to develop a library supporting the representation and use of weighted finite-state representations of speech-recognition models. Using the library, each weighted language or transd... |

105 |
Context-dependent phonetic hidden Markov models for speaker-independent continuous speech recognition
- Lee
- 1990
(Show Context)
Citation Context ...h-recognition systems rely on a variety of probabilistic finite-state models, for instance n-gram language models [21], multiple-pronunciation dictionaries [11], and context-dependent acoustic models =-=[10]-=-. However, most speech recognizers do not take advantage of the shared properties of the information sources they use. Instead, they rely on special-purpose algorithms for specific representations. Th... |

88 |
STL Tutorial and Reference Guide
- Musser, Saini
- 1996
(Show Context)
Citation Context ... then at least all automata classes that support that operation will work. 3 This design philosophy is similar in some ways to other modern software toolkits such as the C++ Standard Template Library =-=[19]-=-. The restricted set of core operations above was motivated by the need to support on-demand implementations of algorithms. In particular, all of those operations are local if we accept the convention... |

84 | Weighted automata in text and speech processing
- Mohri, Pereira, et al.
- 1996
(Show Context)
Citation Context ...is determined by the state's accepting weight. The library includes: Rational operations: union, concatenation, Kleene closure, reversal, inversion and projection; Composition: transducer composition =-=[17]-=-, and acceptor intersection, as well as taking the difference between a weighted acceptor and an unweighted DFA; Equivalence transformations: ffl-elimination, determinization [14, 15] and minimization... |

76 | An efficient compiler for weighted rewrite rules
- Mohri, Sproat
- 1996
(Show Context)
Citation Context ... states in the edit transducer encode appropriate context conditions. Furthermore, a set of weighted edit rules like those in Figure 4 can be directly compiled into an appropriate weighted transducer =-=[18]-=-. Baseform(s) Phone(s) Weights Type a i b j w(a i ; b j ) p pcl pr 1 expansion eh m em 3 contraction r eh ax r 2 transposition t=V 0 V dx 0 context-dependency Fig. 4. Generalized Weighted Edit Distanc... |

54 |
Minimisation of acyclic deterministic automata in linear time, Theoret
- Revuz
- 1992
(Show Context)
Citation Context ...d acceptor and an unweighted DFA; Equivalence transformations: ffl-elimination, determinization [14, 15] and minimization for unweighted (both the general case [1] and the more efficient acyclic case =-=[20]-=-) and weighted acceptors and transducers [12, 15], removal of inaccessible states and transitions; Search: best path, n-best paths, pruning (remove all states and transitions that occur only on paths ... |

50 | On Some Applications of Finite-State Automata Theory to Natural Language
- Mohri
(Show Context)
Citation Context ...ansducer composition [17], and acceptor intersection, as well as taking the difference between a weighted acceptor and an unweighted DFA; Equivalence transformations: ffl-elimination, determinization =-=[14, 15]-=- and minimization for unweighted (both the general case [1] and the more efficient acyclic case [20]) and weighted acceptors and transducers [12, 15], removal of inaccessible states and transitions; S... |

36 |
Digital images and formal languages
- Culik, Kari
- 1997
(Show Context)
Citation Context ...mpractical for sparse DFAs if the symbol set is large, as is the case in language models for large-vocabulary recognition. 1 Weighted acceptors and transducers have also been used in image processing =-=[5]-=-. 1.2 Coverage The library operates on weighted transducers; weighted acceptors are represented as restrictions of the identity transducer to the support of the acceptor. In our chosen representation,... |

36 |
Limited subsets of a free monoid
- Simon
- 1978
(Show Context)
Citation Context ... The correctness of this implementation of minimum edit distance alignment depends on the use of suitable weight combination rules in automata composition, specifically those of the tropical semiring =-=[24]-=-, which will be discussed more fully in the next section. Alignment by transduction can be readily extended to situations in which edits involve longer strings or are context-dependent, as those shown... |

23 | Minimization of Sequential Transducers
- Mohri
- 1994
(Show Context)
Citation Context ... transformations: ffl-elimination, determinization [14, 15] and minimization for unweighted (both the general case [1] and the more efficient acyclic case [20]) and weighted acceptors and transducers =-=[12, 15]-=-, removal of inaccessible states and transitions; Search: best path, n-best paths, pruning (remove all states and transitions that occur only on paths of weight greater by a given threshold than the b... |

19 |
Non Deterministic Stochastic Language Models for Speech Recognition
- Riccardi, Bocchieri, et al.
- 1995
(Show Context)
Citation Context ...s, such as word segmentation for Chinese text [25]. 1.1 Design Rationale Current speech-recognition systems rely on a variety of probabilistic finite-state models, for instance n-gram language models =-=[21]-=-, multiple-pronunciation dictionaries [11], and context-dependent acoustic models [10]. However, most speech recognizers do not take advantage of the shared properties of the information sources they ... |

16 | Syntactic analysis by local grammars automata: an efficient algorithm
- Mohri
- 1994
(Show Context)
Citation Context ...ammar optimization techniques based on it. In some other areas of language processing, especially dictionaries, morphology and local parsing, finite-state techniques have been used with great success =-=[7, 13]-=-. Complex finite-state mappings between strings can be represented with regular expressions over elementary transductions (input-output mappings) together with relational composition of transductions.... |

13 |
Finite growth models
- Ristad, Yianilos
- 1996
(Show Context)
Citation Context ... tr Fig. 1. String Alignment Baseform Phone Weights Type a i b j w(a i ; b j ) ae eh 1 substitution d ffl 2 deletion ffl pr 1 insertion Fig. 2. Weighted Edit Distance suitable machine-learning method =-=[11, 22]-=-. The minimum edit distance between two strings can be simply defined by the recurrences d(a 0 ; b 0 ) = 0 d s (a i ; b j ) = d(a i\Gamma1 ; b j \Gamma1 ) + w(a i ; b j ) (substitution) d d (a i ; b j... |

10 |
Optimal Speech Recognition Using Phone Recognition
- Ljolje, Riley
- 1992
(Show Context)
Citation Context ...ext [25]. 1.1 Design Rationale Current speech-recognition systems rely on a variety of probabilistic finite-state models, for instance n-gram language models [21], multiple-pronunciation dictionaries =-=[11]-=-, and context-dependent acoustic models [10]. However, most speech recognizers do not take advantage of the shared properties of the information sources they use. Instead, they rely on special-purpose... |

7 |
Rational series and their languages (Springer-Verlag
- BERSTEL, REUTENAUER
- 1988
(Show Context)
Citation Context ..., and thus with finite automata with weighted transitions. 1 Weighted languages are just formal power series over appropriate semirings, and weighted transductions can be interpreted in a similar way =-=[23, 6, 9, 3]-=-. We were thus led to develop a library supporting the representation and use of weighted finite-state representations of speech-recognition models. Using the library, each weighted language or transd... |

4 |
The Replace Operator. In 33rd Annual Meeting of the Association for Computational Linguistics
- Karttunen
- 1995
(Show Context)
Citation Context ...ons (input-output mappings) together with relational composition of transductions. Convenient notation extending regular expressions with composition and useful derived forms have also been developed =-=[8]-=-. It might thus be thought that those finite-state tools would apply directly to the problem of specifying and combining the multiple information sources used in a speech recognizer. However, in speec... |

2 |
A general framework for shortest distance problems
- Mohri
- 1997
(Show Context)
Citation Context ...y a regular expression equivalent to a given automaton. In a similar way we defined a general framework for single-source shortestpaths algorithms based on semirings that leads to a generic algorithm =-=[16]-=-. This generic algorithm computes the single-source shortest distance when weights are numbers, strings, or subsets of a set. These different cases are useful in computing minimal deterministic weight... |