Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers

DMCA

Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers (2000)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Erin L. Allwein , Robert E. Schapire , Yoram Singer

Venue:	JOURNAL OF MACHINE LEARNING RESEARCH
Citations:	561 - 20 self

BibTeX

@ARTICLE{Allwein00reducingmulticlass,
    author = {Erin L. Allwein and Robert E. Schapire and Yoram Singer},
    title = {Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers},
    journal = {JOURNAL OF MACHINE LEARNING RESEARCH},
    year = {2000},
    volume = {1},
    pages = {113--141}
}

OpenURL

Abstract

We present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a margin-based binary learning algorithm. The proposed framework unifies some of the most popular approaches in which each class is compared against all others, or in which all pairs of classes are compared to each other, or in which output codes with error-correcting properties are used. We propose a general method for combining the classifiers generated on the binary problems, and we prove a general empirical multiclass loss bound given the empirical loss of the individual binary learning algorithms. The scheme and the corresponding bounds apply to many popular classification learning algorithms including support-vector machines, AdaBoost, regression, logistic regression and decision-tree algorithms. We also give a multiclass generalization error analysis for general output codes with AdaBoost as the binary learner. Experimental results with SVM and AdaBoost show that our scheme provides a viable alternative to the most commonly used multiclass algorithms.

Keyphrases

margin classifier unifying approach binary problem multiclass categorization problem empirical loss viable alternative general method individual binary learning algorithm multiclass generalization error analysis binary learner popular approach logistic regression many popular classification experimental result error-correcting property unifying framework corresponding bound output code margin-based binary learning algorithm general empirical multiclass loss bound decision-tree algorithm used multiclass algorithm general output code support-vector machine

DMCA