Results 1  10
of
49
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2000
"... We present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a marginbased binary learning algorithm. The proposed framework unifies some of the most popular approaches in which each class ..."
Abstract

Cited by 561 (20 self)
 Add to MetaCart
We present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a marginbased binary learning algorithm. The proposed framework unifies some of the most popular approaches in which each class is compared against all others, or in which all pairs of classes are compared to each other, or in which output codes with errorcorrecting properties are used. We propose a general method for combining the classifiers generated on the binary problems, and we prove a general empirical multiclass loss bound given the empirical loss of the individual binary learning algorithms. The scheme and the corresponding bounds apply to many popular classification learning algorithms including supportvector machines, AdaBoost, regression, logistic regression and decisiontree algorithms. We also give a multiclass generalization error analysis for general output codes with AdaBoost as the binary learner. Experimental results with SVM and AdaBoost show that our scheme provides a viable alternative to the most commonly used multiclass algorithms.
Error limiting reductions between classification tasks
 In Proceedings of the International Conference on Machine Learning (ICML
, 2005
"... We introduce a reductionbased model for analyzing supervised learning tasks. We use this model to devise a new reduction from multiclass costsensitive classification to binary classification with the following guarantee: If the learned binary classifier has error rate at most ɛ then the costsens ..."
Abstract

Cited by 45 (7 self)
 Add to MetaCart
(Show Context)
We introduce a reductionbased model for analyzing supervised learning tasks. We use this model to devise a new reduction from multiclass costsensitive classification to binary classification with the following guarantee: If the learned binary classifier has error rate at most ɛ then the costsensitive classifier has cost at most 2ɛ times the expected sum of costs of all possible lables. Since costsensitive classification can embed any bounded loss finite choice supervised learning task, this result shows that any such task can be solved using a binary classification oracle. Finally, we present experimental results showing that our new reduction outperforms existing algorithms for multiclass costsensitive learning. 1
Active classification based on value of classifier
 In NIPS
, 2011
"... Abstract Modern classification tasks usually involve many class labels and can be informed by a broad range of features. Many of these tasks are tackled by constructing a set of classifiers, which are then applied at test time and then pieced together in a fixed procedure determined in advance or a ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
(Show Context)
Abstract Modern classification tasks usually involve many class labels and can be informed by a broad range of features. Many of these tasks are tackled by constructing a set of classifiers, which are then applied at test time and then pieced together in a fixed procedure determined in advance or at training time. We present an active classification process at the test time, where each classifier in a large ensemble is viewed as a potential observation that might inform our classification process. Observations are then selected dynamically based on previous observations, using a valuetheoretic computation that balances an estimate of the expected classification gain from each observation as well as its computational cost. The expected classification gain is computed using a probabilistic model that uses the outcome from previous observations. This active classification process is applied at test time for each individual test instance, resulting in an efficient instancespecific decision path. We demonstrate the benefit of the active scheme on various realworld datasets, and show that it can achieve comparable or even higher classification accuracy at a fraction of the computational costs of traditional methods.
An analytical method for multiclass molecular cancer classification
 SIAM Review
, 2003
"... contributed equally to this work. & & Corresponding author. ..."
Abstract

Cited by 21 (2 self)
 Add to MetaCart
(Show Context)
contributed equally to this work. & & Corresponding author.
Multiclass boosting with repartitioning
 In: Proc. 23rd Int. Conf. Machine Learning, Pittsburgh
, 2006
"... A multiclass classification problem can be reduced to a collection of binary problems with the aid of a coding matrix. The quality of the final solution, which is an ensemble of base classifiers learned on the binary problems, is affected by both the performance of the base learner and the errorcor ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
(Show Context)
A multiclass classification problem can be reduced to a collection of binary problems with the aid of a coding matrix. The quality of the final solution, which is an ensemble of base classifiers learned on the binary problems, is affected by both the performance of the base learner and the errorcorrecting ability of the coding matrix. A coding matrix with strong errorcorrecting ability may not be overall optimal if the binary problems are too hard for the base learner. Thus a tradeoff between errorcorrecting and base learning should be sought. In this paper, we propose a new multiclass boosting algorithm that modifies the coding matrix according to the learning ability of the base learner. We show experimentally that our algorithm is very efficient in optimizing the multiclass margin cost, and outperforms existing multiclass algorithms such as AdaBoost.ECC and onevsone. The improvement is especially significant when the base learner is not very powerful. 1.
Online MultiClass LPBoost
"... Online boosting is one of the most successful online learning algorithms in computer vision. While many challenging online learning problems are inherently multiclass, online boosting and its variants are only able to solve binary tasks. In this paper, we present Online MultiClass LPBoost (OMCLP) ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
Online boosting is one of the most successful online learning algorithms in computer vision. While many challenging online learning problems are inherently multiclass, online boosting and its variants are only able to solve binary tasks. In this paper, we present Online MultiClass LPBoost (OMCLP) which is directly applicable to multiclass problems. From a theoretical point of view, our algorithm tries to maximize the multiclass softmargin of the samples. In order to solve the LP problem in online settings, we perform an efficient variant of online convex programming, which is based on primaldual gradient descentascent update strategies. We conduct an extensive set of experiments over machine learning benchmark datasets, as well as, on Caltech101 category recognition dataset. We show that our method is able to outperform other online multiclass methods. We also apply our method to tracking where, we present an intuitive way to convert the binary tracking by detection problem to a multiclass problem where background patterns which are similar to the target class, become virtual classes. Applying our novel model, we outperform or achieve the stateoftheart results on benchmark tracking videos.
Boosting multiclass learning with repeating codes
, 2006
"... Motivation: Determining locations of protein expression is essential to understand protein function. Advances in green fluorescence protein (GFP) fusion proteins and automated fluorescence microscopy allow for rapid acquisition of large collections of protein localization images. Recognition of thes ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Motivation: Determining locations of protein expression is essential to understand protein function. Advances in green fluorescence protein (GFP) fusion proteins and automated fluorescence microscopy allow for rapid acquisition of large collections of protein localization images. Recognition of these cell images requires an automated image analysis system. Approaches taken by previous work concentrated on designing a set of optimal features and then applying standard machine learning algorithms. In fact, trends of recent advances in machine learning and computer vision can be applied to improve the performance. One trend is the advances in multiclass learning with errorcorrecting output codes (ECOC). Another trend is the use of a large number of weak detectors with boosting for detecting objects in images of realworld scenes. Results: We take advantage of these advances to propose a new
Machine Learning Techniques—Reductions Between Prediction Quality Metrics
"... Abstract Machine learning involves optimizing a loss function on unlabeled data points given examples of labeled data points, where the loss function measures the performance of a learning algorithm. We give an overview of techniques, called reductions, for converting a problem of minimizing one los ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
Abstract Machine learning involves optimizing a loss function on unlabeled data points given examples of labeled data points, where the loss function measures the performance of a learning algorithm. We give an overview of techniques, called reductions, for converting a problem of minimizing one loss function into a problem of minimizing another, simpler loss function. This tutorial discusses how to create robust reductions that perform well in practice. The reductions discussed here can be used to solve any supervised learning problem with a standard binary classification or regression algorithm available in any machine learning toolkit. We also discuss common design flaws in folklore reductions. 1
Multiclass Boosting with Hinge Loss based on Output Coding
"... Multiclass classification is an important and fundamental problem in machine learning. A popular family of multiclass classification methods belongs to reducing multiclass to binary based on output coding. Several multiclass boosting algorithms have been proposed to learn the coding matrix and the a ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
Multiclass classification is an important and fundamental problem in machine learning. A popular family of multiclass classification methods belongs to reducing multiclass to binary based on output coding. Several multiclass boosting algorithms have been proposed to learn the coding matrix and the associated binary classifiers in a problemdependent way. These algorithms can be unified under a sumofexponential loss function defined in the domain of margins (Sun et al., 2005). Instead, multiclass SVM uses another type of loss function based on hinge loss. In this paper, we present a new outputcodingbased multiclass boosting algorithm using the multiclass hinge loss, which we call HingeBoost.OC. HingeBoost.OC is tested on various real world datasets and shows better performance than the existing multiclass boosting algorithm AdaBoost.ERP, onevsone, onevsall, ECOC and multiclass SVM in a majority of different cases. 1.
Combined Binary Classifiers With Applications To Speech Recognition
 NEARESTNEIGHBOR ECOC WITH APPLICATION TO ALLPAIRS MULTICLASS SVM
, 2002
"... Many applications require classification of examples into one of several classes. A common way of designing such classifiers is to determine the class based on the outputs of several binary classifiers. We consider some of the most popular methods for combining the decisions of the binary classifier ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
Many applications require classification of examples into one of several classes. A common way of designing such classifiers is to determine the class based on the outputs of several binary classifiers. We consider some of the most popular methods for combining the decisions of the binary classifiers, and improve existing bounds on the error rates of the combined classifier over the training set. We also describe a new method for combining binary classifiers. The method is based on stacking a neural network and, when used with support vector machines as the binary learners, substantially decreased the error rate in two vowel classification tasks.