## A Maximum Entropy Approach to Adaptive Statistical Language Modeling (1996)

### Cached

### Download Links

Venue: | Computer, Speech and Language |

Citations: | 242 - 11 self |

### BibTeX

@ARTICLE{Rosenfeld96amaximum,

author = {Ronald Rosenfeld},

title = {A Maximum Entropy Approach to Adaptive Statistical Language Modeling},

journal = {Computer, Speech and Language},

year = {1996},

volume = {10},

pages = {187--228}

}

### Years of Citing Articles

### OpenURL

### Abstract

An adaptive statistical languagemodel is described, which successfullyintegrates long distancelinguistic information with other knowledge sources. Most existing statistical language models exploit only the immediate history of a text. To extract information from further back in the document's history, we propose and use trigger pairs as the basic information bearing elements. This allows the model to adapt its expectations to the topic of discourse. Next, statistical evidence from multiple sources must be combined. Traditionally, linear interpolation and its variants have been used, but these are shown here to be seriously deficient. Instead, we apply the principle of Maximum Entropy (ME). Each information source gives rise to a set of constraints, to be imposed on the combined estimate. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the ME solution...