Citation Context

...st set. The ‘ModApte’ train/test split of the Reuters 21578 Distribution 1.0 data set consists of 12902 Reuters newswire articles in 135 overlapping topic categories. Following several other studies [=-=Joachims 1998-=-; Liere & Tadepalli 1997] we build binary classifiers for each of the 10 most populous classes. We ignore words on a stoplist, but do not use stemming. The resulting vocabulary has 19371 words. Result...

Citation Context

...ormulation of naive Bayes assumes a multinomial event model for documents; this generally produces better text classification accuracy than another formulation that assumes a multi-variate Bernoulli [=-=McCallum & Nigam 1998-=-]. 3 EM and Unlabeled Data When naive Bayes is given just a small set of labeled training data, classification accuracy will suffer because variance in the parameter estimates of the generative model ...

Citation Context

...4; Craven et al. 1998; Joachims 1997]. This paradox is explained by the fact that classification estimation is only a function of the sign (in binary cases) of the function estimation [Friedman 1997; =-=Domingos & Pazzani 1997-=-]. Also note that our formulation of naive Bayes assumes a multinomial event model for documents; this generally produces better text classification accuracy than another formulation that assumes a mu...

Citation Context

...ith a Normal distribution. Several other studies have investigated active learning for text categorization. Lewis and Gale examine uncertainty sampling and relevance sampling in a pool-based setting [=-=Lewis & Gale 1994-=-; Lewis 1995]. These techniques select queries based on only a single classifier instead of a committee, and thus cannot approximate classification variance. Liere and Tadepalli [1997] use committees ...

Citation Context

... each class would have generated the test document in question, and selecting the most probable class. Our parametric model is naive Bayes, which is based on commonly used assumptions [Friedman 1997; =-=Joachims 1997-=-]. First we assume that text documents are generated by a mixture model (parameterized by θ), and that there is a one-to-one correspondence between the (observed) class labels and the mixture componen...

Citation Context

... Note that our assumptions about the generation of text documents are all violated in practice, and yet empirically, naive Bayes does a good job of classifying text documents [Lewis & Ringuette 1994; =-=Craven et al. 1998-=-; Joachims 1997]. This paradox is explained by the fact that classification estimation is only a function of the sign (in binary cases) of the function estimation [Friedman 1997; Domingos & Pazzani 19...

Citation Context

...fication error and variance over the distribution of examples [Cohn, Ghahramani, & Jordan 1996]. When calculating this in closed-form is prohibitively complex, the Query-by-Committee (QBC) algorithm [=-=Freund et al. 1997-=-] can be used to select documents that have high classification variance themselves. QBC measures the variance indirectly, by examining the disagreement among class labels assigned by a set of classif...

Citation Context

... arg maxj P(cj|di; ˆ θ). Note that our assumptions about the generation of text documents are all violated in practice, and yet empirically, naive Bayes does a good job of classifying text documents [=-=Lewis & Ringuette 1994-=-; Craven et al. 1998; Joachims 1997]. This paradox is explained by the fact that classification estimation is only a function of the sign (in binary cases) of the function estimation [Friedman 1997; D...

Citation Context

...robability that each class would have generated the test document in question, and selecting the most probable class. Our parametric model is naive Bayes, which is based on commonly used assumptions [=-=Friedman 1997-=-; Joachims 1997]. First we assume that text documents are generated by a mixture model (parameterized by θ), and that there is a one-to-one correspondence between the (observed) class labels and the m...

Citation Context

...antly improved classification accuracy on the test set when the pool of labeled examples is small [Nigam et al. 1998]. 1 This use of EM is a special case of a more general missing values formulation [=-=Ghahramani & Jordan 1994-=-]. In implementation, EM is an iterative two-step process. The E-step calculates probabilistically-weighted class labels, P(cj|di; ˆ θ), for every unlabeled document using a current estimate of θ and ...

Citation Context

...ctive learning with Expectation-Maximization (EM) in order to take advantage of the word co-occurrence information contained in the many documents that remain in the unlabeled pool. In previous work [=-=Nigam et al. 1998-=-] we show that combining the evidence of labeled and unlabeled documents via EM can reduce text classification error by one-third. We treat the absent labels as “hidden variables” and use EM to fill t...

Citation Context

..., QBC should pick more informative documents to label. The complete active learning algorithm, both with and without EM, is summarized in Table 1. Unlike settings in which queries must be generated [=-=Cohn 1994-=-], and previous work in which the unlabeled data is available as a stream [Dagan & Engelson 1995; Liere & Tadepalli 1997; Freund et al. 1997], our assumption about the availability of a pool of unlabe...

Citation Context

...lassifier k times, resulting in k committee members. Individual committee members are denoted by m. We consider two metrics for measuring committee disagreement. The previously employed vote entropy [=-=Dagan & Engelson 1995-=-] is the entropy of the class label distribution resulting from having each committee member “vote” with probability mass 1/k for its winning class. One disadvantage of vote entropy is that it does no...

Citation Context

...ces text classification error by one-third [Nigam et al. 1998]. Two other studies have used EM to combine labeled and unlabeled data without active learning for classification, but on non-text tasks [=-=Miller & Uyar 1997-=-; Shahshahani & Landgrebe 1994]. Ghahramani and Jordan [1994] use EM with mixture models to fill in missing feature values. 6 Experimental Results This section provides evidence that using a combinati...

Citation Context

...can be a computational convenience.) We consider three ways of selecting documents: stream-based, pool-based, and densityweighted pool-based. Some previous applications of QBC [Dagan & Engelson 1995; =-=Liere & Tadepalli 1997-=-] use a simulated stream of unlabeled documents. When a document is produced by the stream, this approach measures the classification disagreement among the committee members, and decides, based on th...