DMCA
Using Maximum Entropy for Text Classification (1999)
Cached
Download Links
| Citations: | 325 - 6 self |
BibTeX
@MISC{Nigam99usingmaximum,
author = {Kamal Nigam and John Lafferty and Andrew Mccallum},
title = {Using Maximum Entropy for Text Classification},
year = {1999}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper proposes the use of maximum entropy techniques for text classification. Maximum entropy is a probability distribution estimation technique widely used for a variety of natural language tasks, such as language modeling, part-of-speech tagging, and text segmentation. The underlying principle of maximum entropy is that without external knowledge, one should prefer distributions that are uniform. Constraints on the distribution, derived from labeled training data, inform the technique where to be minimally non-uniform. The maximum entropy formulation has a unique solution which can be found by the improved iterative scaling algorithm. In this paper, maximum entropy is used for text classification by estimating the conditional distribution of the class variable given the document. In experiments on several text datasets we compare accuracy to naive Bayes and show that maximum entropy is sometimes significantly better, but also sometimes worse. Much future work remains, but the re...
Keyphrases
maximum entropy text classification part-of-speech tagging probability distribution estimation technique much future work improved iterative scaling algorithm labeled training data external knowledge naive bayes maximum entropy technique natural language task several text datasets underlying principle conditional distribution text segmentation maximum entropy formulation language modeling unique solution
