Abstract

The effect of selecting varying numbers and kinds of features for use in predicting category membership was investigated on the Reuters and MUC-3 text categorization data sets. Good categorization performance was achieved using a statistical classifier and a proportional assignment strategy. The optimal feature set size for word-based indexing was found to be surprisingly low (10 to 15 features) despite the large training sets. The extraction of new text features by syntactic analysis and feature clustering was investigated on the Reuters data set. Syntactic indexing phrases, clusters of these phrases, and clusters of words were all found to provide less effective representations than individual words. 1. Introduction Text categorization---the automated assigning of natural language texts to predefined categories based on their content---is a task of increasing importance. Its applications include indexing texts to support document retrieval [1], extracting data from texts [2], and ai...

Citations

649	A stochastic parts program and noun phrase parser for unrestricted text - Church - 1988
152	Coding and Information Theory - Hamming - 1980
141	Representation and Learning in Information Retrieval - Lewis - 1992
136	Pattern Classification and Scene Analysis; Wiley-Interscience - Duda, Hart - 1973
78	Models for retrieval with probabilistic indexing - Fuhr - 1989
76	Evaluating text categorization - Lewis - 1991
60	CONSTRUE/TIS: a system for content-based indexing of a database of news stories - Hayes, Weinstein - 1990
56	Term clustering of syntactic phrases - Lewis, Croft - 1989
46	Air/x - a rule-based multistage indexing system for large subject fields - Fuhr, Hartmann, et al. - 1991
7	Data Extraction as Text Categorization: An Experiment with the MUC-3 Corpus - Lewis - 1991