Abstract

A number of feature selection metrics have been explored in text categorization, among which information gain (IG), chi-square (CHI), correlation coefficient (CC) and odds ratios (OR) are considered most effective. CC and OR are one-sided metrics while IG and CHI are two-sided. Feature selection using one-sided metrics selects the features most indicative of membership only, while feature selection using two-sided metrics implicitly combines the features most indicative of membership (e.g. positive features) and nonmembership (e.g. negative features) by ignoring the signs of features. The former never consider the negative features, which are quite valuable, while the latter cannot ensure the optimal combination of the two kinds of features especially on imbalanced data. In this work, we investigate the usefulness of explicit control of that combination within a proposed feature selection framework. Using multinomial naïve Bayes and regularized logistic regression as classifiers, our experiments show both great potential and actual merits of explicitly combining positive and negative features in a nearly optimal fashion according to the imbalanced data. 1.

Citations

839	Machine learning in automated text categorization - Sebastiani - 2002
775	G: Wrappers for Feature Subset Selection - Kohavi, John - 1996
739	A Comparative Study on Feature Selection in Text Categorization - Yang
619	Nigam K: A comparison of event models for naïve Bayes text classification - McCallum - 1998
419	Inductive learning algorithms and representations for text categorization - Dumais, Platt, et al. - 1998
413	An Evaluation of Statistical Approaches to Text Categorization - Yang - 1999
239	Comparison of Two Learning Algorithms for Text Categorization - Lewis, Ringuette - 1994
180	An Extensive Empirical Study of Feature Selection Metrics for Text Classification - Forman
110	S: The class imbalance problem: A systematic study. Intelligent Data Analysis 2002 - Japkowicz, Stephen
100	Feature Selection, Perceptron Learning, and a Usability Case Study for Text Categorization - Ng, Goh, et al. - 1997
67	Text categorization based on regularized linear classification methods - Zhang, Oles - 2001
50	Learning routing queries in a query zone - Singhal, Mitra, et al. - 1997
44	Adaptive fraud detection," Data mining and knowledge discovery (1:3 - Fawcett, Provost - 1997
19	Robustness of regularized linear classification methods in text categorization - Zhang, Yang
2	Feture selection for unbalanced class distribution and naive bayes - Mladeni, Marko - 1999
1	Machine Learning on non-homogeneous, distributed text data - Mladeni - 1998