Abstract

Learners based on different paradigms can be combined for improved accuracy. Each learning method assumes a certain model that comes with a set of assumptions which may lead to error if the assumptions do not hold. Learning is an ill-posed problem and with finite data each algorithm converges to a different solution and fails under different circumstances. Our previous experience with statistical and neural classifiers was that classifiers based on these paradigms do generalize differently, fail on different patterns and to a certain extent complement each other and thus we look for ways to combine them for higher accuracy. One way to get complementary classifiers is by using different input representations. The methods we investigate are voting, mixture of experts, stacking and cascading. We do experiments on three real-world applications: optical handwritten digit recognition, pen-based handwritten digit recognition and the estimation of road travel distances which is a regression pr...

Citations

1998	Bagging predictors - Breiman - 1996
1325	Experiments with a new boosting algorithm - Freund, Schapire - 1996
665	Adaptive mixtures of local experts - Jacobs, Jordan, et al. - 1991
634	Hierarchical mixtures of experts and the EM algorithm - Jordan, Jacobs - 1994
463	Stacked generalization - Wolpert - 1992
425	Neural network ensembles - Hansen, Salamon - 1990
354	Neural network ensembles, cross validation, and active learning - Krogh, Vedelsby - 1995
284	C.Y.: Several methods for combining multiple classifiers and their applications in handwritten character recognition - Xu, Krzyzak, et al. - 1992
217	K.: Neural-gas network for vector quantization and its application to time-series prediction - Martinetz, Schulten - 1993
121	Methods for combining experts’ probability assessments - Jacobs - 1995
114	Error Reduction through Learning Multiple Descriptions - Ali, Pazzani - 1996
94	Combining the Results of Several Neural-Network Classifiers. Neural Networks 7: 777–781 - Rogova - 1994
75	L.: Computer recognition of unconstrained handwritten numerals. Procs - Suen, Nadal, et al. - 1992
70	Improving Regression Estimation: Averaging methods for variance reduction with extensions to general convex measure optimization - Perrone - 1993
66	Combining estimates on regression and classification - LeBlanc, Tibshirani - 1996
66	Hybrid system for protein secondary structure prediction - Zhang, Mesirov, et al. - 1992
54	Synergy of clustering multiple back-propagation networks - Lincoln, Skrzypek - 1990
42	Consensus theoretic classification methods - Benediktsson, Swain - 1992
32	Exploiting neurons with localized receptive fields to learn chaos - Stokbro, Umberger, et al. - 1990
25	Theoretical foundations of linear and order statistics combiner for neural pattern classifiers - Tumer, Ghosh - 1995
23	Multiple networks for function learning - Alpaydin - 1993
23	Local linear perceptrons for classification - Alpaydin, Jordan - 1996
22	Voting over multiple condensed nearest neighbors - Alpaydin - 1997
21	Lowering variance of decisions by using artificial neural network portfolios - Mani - 1991
15	Multistage pattern recognition with reject option - Pudil, Novovicova, et al. - 1992
15	High-performance reading machines - Srihari - 1992
12	Cascading classifiers - Alpaydın, Kaynak - 1998
10	Bias, variance and the combination of estimators; the case of linear least squares - Meir - 1994
8	Combining multiple classifiers for pen-based handwritten digit recognition,” M.Sc - Alimo˘glu - 1996
7	Methods of combining multiple classifiers and their applications to handwritten digit recognition. Unpublished master’s thesis - Kaynak - 1995
2	Parametric Distance Functions vs. Nonparametric Neural Networks for Estimating Road Travel Distances - ALPAYDIN, ALTINEL, et al. - 1996
1	REx: Learning A Rule and - Alpaydin - 1997