We develop a common theoretical framework for combining classifiers which use distinct pattern representations and show that many existing schemes can be considered as special cases of compound classification where all the pattern representations are used jointly to make a decision. An experimental comparison of various classifier combination schemes demonstrates that the combination rule developed under the most restrictive assumptions—the sum rule—outperforms other classifier combinations schemes. A sensitivity analysis of the various schemes to estimation errors is carried out to show that this finding can be justified theoretically.

Multiple classifier fusion may generate more accurate classification than each of the constituent classifiers. Fusion is often based on fixed combination rules like the product and average. Only under strict probabilistic conditions can these rules be justified. We present here a simple rule for adapting the class combiner to the application. c decision templates (one per class) are estimated with the same training set that is used for the set of classifiers. These templates are then matched to the decision profile of new incoming objects by some similarity measure. We compare 11 versions of our model with 14 other techniques for classifier fusion on the Satimage and Phoneme datasets from the database ELENA. Our results show that decision templates based on integral type measures of similarity are superior to the other schemes on both data sets.

We suggest two simple ways to use a genetic algorithm (GA) to design a multiple classifier system. The first GA version selects disjoint feature subsets to be used by the individual classifiers, whereas the second version selects (possibly) overlapping feature subsets and also the types of the individual classifiers. The two GAs have been tested with four real data sets: Heart, Satimage, Letters, and Forensic glasses (10-fold cross-validation, except for Satimage where we used only two splits). We used 3-classifier systems and basic types of individual classi ers (the linear and quadratic discriminant classifiers and the logistic classifier). The multiple classifier systems designed with the two GAs were compared against classi ers using: (a) all features; (b) the best feature subset found by the sequential backward selection (SBS) method; and (c), the best feature subset found by a GA (individual classifier!). We found that: (1) the multiple classifier system derived through the GA, Version 2, ...

This paper gives an overview of classifier fusion methods and attempts to identify new trends that may dominate this area of research in future. A taxonomy of fusion methods trying to bring some order into the existing "pudding of diversities" is also provided

Current-generation automatic speech recognition #ASR# systems assume that words are readily decomposable into constituent phonetic components ##phonemes"#. A detailed linguistic dissection of state-of-the-art speech recognition systems indicates that the conventional phonemic #beads-on-a-string" approach is of limited utility, particularly with respect to informal, conversational material. The study shows that there is a signi#cantgapbetween the observed data and the pronunciation models of current ASR systems. It also shows that many important factors a#ecting recognition performance are not modeled explicitly in these systems.

Abstract not found

Boosting is recognized as one of the most successful techniques for generating classifier ensembles. Typically, the classifier outputs are combined by the weighted majority vote. The purpose of this study is to demonstrate the advantages of some fuzzy combination methods for ensembles of classifiers designed by Boosting. We ran 2-fold cross-validation experiments on 6 benchmark data sets to compare the fuzzy and non-fuzzy combination methods. On the "fuzzy side" we used the fuzzy integral and the decision templates with different similarity measures. On the "non-fuzzy side" we tried simple combiners such as the majority vote, minimum, maximum, average, product, and the Naive Bayes combination. Surprisingly, the minimum, maximum, average and product, which have been reported elsewhere to work very well on a variety of problems, appeared to be inadequate for our task. Thus the real contest was among the fuzzy combination methods on the one hand, and the weighted majority vote, the simple majority vote, and the Naive Bayes combiner, on the other hand. In our experiments, the fuzzy methods performed consistently better than the nonfuzzy methods. The weighted majority vote showed a stable performance, though slightly inferior to the performance of the fuzzy combiners. The majority vote and the Naive Bayes combiners had erratic behavior, ranging from the best to the worst contestants for different data sets.

Abstract ∗ Classifier combination is now an established pattern recognition subdiscipline. Despite the strong aspiration for theoretical studies, classifier combination relies mainly on heuristic and empirical solutions. Assuming that “soft computing ” encompasses neural networks, evolutionary computation, and fuzzy sets, we explain how each of the three components has been used in classifier combination. Let D = {D1, D2,..., DL} be a set of classifiers (we shall also call D a team or ensemble), and let Ω = {ω1,..., ωc} be a set of class labels. Each classifier gets as its input a feature vector x = [x1,..., xn] T, x ∈ ℜ n and assigns it to a class label from Ω, i.e., Di: ℜ n → Ω. Alternatively, we may define the classifier

vi Table of Contents viii List of Figures xiii List of Tables xiv List of Symbols xvii List of Acronyms xx Chapter 1. Introduction 1 Chapter 2. Background and Related Research 8 2.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 2.2 Generalization : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9 2.3 Statistical Background : : : : : : : : : : : : : : : : : : : : : : : : 13 2.4 Regularization : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16 2.5 Motivation for Combining : : : : : : : : : : : : : : : : : : : : : : 18 2.6 Historical sketch : : : : : : : : : : : : : : : : : : : : : : : : : : : 19 viii 2.6.1 Survey of Recent Literature : : : : : : : : : : : : : : : : : 19 2.6.2 Belief and Evidence Combining : : : : : : : : : : : : : : : 22 2.6.3 Economic Forecasting : : : : : : : : : : : : : : : : : : : : 23 2.6.4 Stacked Generalization : : : : : : : : : : : : : : : : : : : : 23 2.6.5 Ensemble Methods : : : : : : : : : : : : : : : : : : : : : ...

This paper presents a new classifier combination technique based on the Dempster-Shafer theory of evidence. The Dempster-Shafer theory of evidence is a powerful method for combining measures of evidence from different classifiers. However, since each of the available methods that estimates the evidence of classifiers has its own limitations, we propose here a new implementation which adapts to training data so that the overall mean square error is minimized. The proposed technique is shown to outperform most available classifier combination methods when tested on three different classification problems.