Abstract

In this paper we describe a new method to reduce the complexity of support vector machines by reducing the number of necessary support vectors included in their solutions. The reduction process iteratively selects two nearest support vectors belonging to the same class and replaces them by a newly constructed vector. Through the analysis of relation between vectors in the input and feature spaces, we present the construction of new vectors that requires to find the unique maximum point of a one-variable function on the interval (0, 1), not to minimize a function of many variables with local minimums in former reduced set methods. Experimental results on real life datasets show that the proposed method is effective in reducing number of support vectors and preserving machine’s generalization performance.

Citations

1656	CJC: A tutorial on support vector machines for pattern recognition - Burges - 1998
1512	Smola A: Learning with Kernels - Schölkopf - 2002
1491	Support-vector networks - Cortes, Vapnik - 1995
1466	Numerical Recipes in C. The Art of Scientific Computing - PRESS, FLANNERY, et al. - 1988
1368	Text categorization with support vector machines - Joachims - 1998
933	A training algorithm for optimal margin classifiers - Boser, Guyon, et al. - 1992
741	Shawe-Taylor J (2000) An Introduction to Support Vector Machines and other kernel-based learning methods - Cristianini
122	Improving the accuracy and speed of support vector learning machines - Burges, Schölkopf - 1997
120	Simplified support vector decision rules - Burges - 1996
94	Kernel PCA and de–noising in feature spaces - Mika, Schölkopf, et al. - 1999
60	Input space versus feature space in kernelbased methods - Scholkopf, Mika, et al. - 1999
35	The pre-image problem in kernel methods - Kwok, Tsang
31	Training support vector machines:an application to face detection - Osuna, Freund, et al. - 1997
28	Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks - LeCun, Jackel, et al. - 1995
28	H.: Handwritten Digit Recognition: Benchmarking of State-of-the-art Techniques - Liu, Nakashima, et al.
7	D.: Fast query-optimized kernel machine classification via incremental approximate nearest support vectors - DeCoste, Mazzoni - 2003