Abstract

Boosting is recognized as one of the most successful techniques for generating classifier ensembles. Typically, the classifier outputs are combined by the weighted majority vote. The purpose of this study is to demonstrate the advantages of some fuzzy combination methods for ensembles of classifiers designed by Boosting. We ran 2-fold cross-validation experiments on 6 benchmark data sets to compare the fuzzy and non-fuzzy combination methods. On the "fuzzy side" we used the fuzzy integral and the decision templates with different similarity measures. On the "non-fuzzy side" we tried simple combiners such as the majority vote, minimum, maximum, average, product, and the Naive Bayes combination. Surprisingly, the minimum, maximum, average and product, which have been reported elsewhere to work very well on a variety of problems, appeared to be inadequate for our task. Thus the real contest was among the fuzzy combination methods on the one hand, and the weighted majority vote, the simple majority vote, and the Naive Bayes combiner, on the other hand. In our experiments, the fuzzy methods performed consistently better than the nonfuzzy methods. The weighted majority vote showed a stable performance, though slightly inferior to the performance of the fuzzy combiners. The majority vote and the Naive Bayes combiners had erratic behavior, ranging from the best to the worst contestants for different data sets.

Citations