Evaluating and Optimizing Autonomous Text Classification Systems (1995) [72 citations — 9 self]
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
author = {David Lewis},
title = {Evaluating and Optimizing Autonomous Text Classification Systems},
booktitle = {},
year = {1995},
pages = {246--254},
publisher = {ACM Press}
}
Years of Citing Articles
Bookmarks
OpenURL
Abstract:
Text retrieval systems typically produce a ranking of documents and let a user decide how far down that ranking to go. In contrast, programs that filter text streams, software that categorizes documents, agents which alert users, and many other IR systems must make decisions without human input or supervision. It is important to define what constitutes good effectiveness for these autonomous systems, tune the systems to achieve the highest possible effectiveness, and estimate how the effectiveness changes as new data is processed. We show how to do this for binary text classification systems, emphasizing that different goals for the system lead to different optimal behaviors. Optimizing and estimating effectiveness is greatly aided if classifiers that explicitly estimate the probability of class membership are used. 1 Introduction Ranked retrieval is the information retrieval (IR) researcher's favorite tool for dealing with information overload. Ranked retrieval systems display docum...
