## Order Statistics Combiners For Neural Classifiers (1995)

Venue: | In Proceedings of the World Congress on Neural Networks |

Citations: | 15 - 7 self |

### BibTeX

@INPROCEEDINGS{Tumer95orderstatistics,

author = {Kagan Tumer and Joydeep Ghosh},

title = {Order Statistics Combiners For Neural Classifiers},

booktitle = {In Proceedings of the World Congress on Neural Networks},

year = {1995},

pages = {31--34},

publisher = {INNS Press}

}

### OpenURL

### Abstract

: Several researchers have shown that linearly combining outputs of multiple neural classifiers results in better performance for many applications. In this paper we introduce a family of order statistics combiners as an alternative to linear combiners. We show analytically that the selection of the median, the maximum and in general, the i th order statistic improves classification performance. Specifically, we show that order statistics combiners reduce the variance of the actual decision boundaries around the optimum boundary, and that this is directly related to classification error. 1 Introduction Training a parametric classifier involves the use of a training set of data with known labeling to estimate or "learn" the parameters of the chosen model. A test set, consisting of patterns not previously seen by the classifier, is then used to determine the classification performance, or generalization ability. The generalization ability of a classifier depends on the selection of t...

### Citations

579 | Stacked generalization
- Wolpert
- 1992
(Show Context)
Citation Context ...ever, selecting the "best" classifier in such fashion is not necessarily the ideal choice, since potentially valuable information may be wasted by discarding the results of lesssuccessful cl=-=assifiers [8]-=-. In order to avoid the potential loss of information through selecting only one classifier, the outputs of all the available classifiers can be pooled before a decision is made. Classifier combining ... |

399 |
Methods of combining multiple classifiers and their applications to handwriting recognition
- Xu, Krzyzak, et al.
- 1992
(Show Context)
Citation Context ...ier, the outputs of all the available classifiers can be pooled before a decision is made. Classifier combining has proved to improve the classification result of a variety of real world applications =-=[2, 3, 9]-=-. Most of the analysis in this area, however, focuses on linear combiners [4, 5]. In this paper we propose a family of Order Statistics (OS) combiners as an alternative to linear combiners. This choic... |

307 | When networks disagree: Ensemble methods for hybrid neural networks
- Perrone, Cooper
- 1993
(Show Context)
Citation Context ...s made. Classifier combining has proved to improve the classification result of a variety of real world applications [2, 3, 9]. Most of the analysis in this area, however, focuses on linear combiners =-=[4, 5]-=-. In this paper we propose a family of Order Statistics (OS) combiners as an alternative to linear combiners. This choice is motivated by the observation that certain classifiers may err significantly... |

282 |
Neural network classifiers estimate Bayesian a posterior probabilities
- Richard, Lippmann
- 1991
(Show Context)
Citation Context ...at the outputs of parametric classifiers trained to minimize mean square error functions, given one-of-L desired outputs, approximate the a posteriori probability densities of the corresponding class =-=[6]-=-, let us express the output response of the i th unit of a classifier network to a given input x as 2 : f i (x) = p i (x) + ffl i (x) , where p i (x) is the a posteriori probability distribution of th... |

167 |
A First Course in Order Statistics
- Arnold, Balakrishnan, et al.
- 1992
(Show Context)
Citation Context ...ing order, providing, X 1:NsX 2:Ns\Delta \Delta \DeltasXN :N . The i th order statistic denoted by X i:N , is the i th value in this progression. The probability density function of X i:N is given by =-=[1]-=-: fX i:N (x) = N ! (i \Gamma 1)! (N \Gamma i)! [F X (x)] i\Gamma1 [1 \Gamma FX (x)] N \Gammai fX (x) : (3) This general form however, cannot always be computed in closed form. However, the first two m... |

33 | Structural adaptation and generalization in supervised feedforward networks
- Gosh, Tumer
- 1994
(Show Context)
Citation Context ...ier, the outputs of all the available classifiers can be pooled before a decision is made. Classifier combining has proved to improve the classification result of a variety of real world applications =-=[2, 3, 9]-=-. Most of the analysis in this area, however, focuses on linear combiners [4, 5]. In this paper we propose a family of Order Statistics (OS) combiners as an alternative to linear combiners. This choic... |

24 | Approximating a function and its derivatives using MSE-optimal linear combinations of trained feedforward neural networks
- Hashem, Schmeiser
- 1993
(Show Context)
Citation Context ...s made. Classifier combining has proved to improve the classification result of a variety of real world applications [2, 3, 9]. Most of the analysis in this area, however, focuses on linear combiners =-=[4, 5]-=-. In this paper we propose a family of Order Statistics (OS) combiners as an alternative to linear combiners. This choice is motivated by the observation that certain classifiers may err significantly... |

15 | Integration of neural classifiers for passive sonar signals
- Ghosh, Tumer, et al.
- 1996
(Show Context)
Citation Context ...ier, the outputs of all the available classifiers can be pooled before a decision is made. Classifier combining has proved to improve the classification result of a variety of real world applications =-=[2, 3, 9]-=-. Most of the analysis in this area, however, focuses on linear combiners [4, 5]. In this paper we propose a family of Order Statistics (OS) combiners as an alternative to linear combiners. This choic... |

6 |
Boundary variance reduction for improved classification through hybrid networks (Invited paper
- Turner, Ghosh
- 1995
(Show Context)
Citation Context ...x, and ffl i (x) is the error associated with the i th output 3 . The decision boundary provided by a given classifier is offset from the ideal boundary by b, given as a first order approximation, by =-=[7]-=-: b = ffl i (x b ) \Gamma ffl j (x b ) s ; (1) where s is a constant representing the difference of the derivatives of each a posteriori density function at the ideal boundary. Let the errors ffl i (x... |