Abstract

We describe a new probabilistic approach in the language modeling framework that captures adaptively the local term dependencies in documents. The new model works by boosting scores of documents that contain topic-specific local dependencies and exhibits the behavior of the unigram model in the absence of such dependencies. Contributions of the current work include adapting van Rijsbergen 's [14] work in the classical probabilistic framework to the language modeling framework and adaptive modeling of withinsentence dependencies.

Citations