A tutorial on support vector machines for pattern recognition

A tutorial on support vector machines for pattern recognition (1998)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Christopher J. C. Burges

Venue:	Data Mining and Knowledge Discovery
Citations:	1656 - 11 self

BibTeX

@ARTICLE{Burges98atutorial,
    author = {Christopher J. C. Burges},
    title = {A tutorial on support vector machines for pattern recognition},
    journal = {Data Mining and Knowledge Discovery},
    year = {1998},
    volume = {2},
    pages = {121--167}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.

Citations

6695	Statistical Learning Theory - Vapnik - 1998
3912	CM: Neural Networks for Pattern Recognition - Bishop - 1995
3200	Topics in Matrix Analysis - Horn, Johnson - 1994
1493	Vapnik V: Support-Vector Networks - Cortes - 1995
1470	Numerical Recipes: The Art of Scientific Computing - Press, Flannery, et al. - 1986
1366	Text categorization with support vector machines: learning with many relevant features - Joachims - 1998
934	A training algorithm for optimal margin classifiers - Boser, Guyon, et al. - 1992
882	Practical Methods of Optimization - Fletcher - 1987
801	A probabilistic theory of pattern recognition - Devroye, Györfi, et al. - 1996
454	Training support vector machines: an application to face detection - Osuna, Freund, et al. - 1997
261	Introduction to Applied Mathematics - Strang - 1986
221	Structural risk minimization over data-dependent hierarchies - Shawe-Taylor, Bartlett, et al. - 1940
208	Rozonoer L: Theoretical foundations of the potential function method in pattern recognition learning. Automat Rem Contr - Aizerman, Braverman - 1964
203	Girosi F: An improved training algorithm for support vector machines - Osuna, Freund - 1997
186	Numerical Recipes in C: The Art of Scientific Computing (Cambridge - Press, Teukolsky, et al. - 1992
168	An Equivalence Between Sparse Approximation and Support Vector Machines. Neural Comput - Girosi - 1998
155	Nonlinear programming - Mangasarian - 1969
154	Extracting support data for a given task - Scholkopf, Burges, et al. - 1995
140	Support vector method for function approximation, regression estimation, and signal processing - Vapnik, Golowich, et al. - 1997
130	LOQO: An interior point code for quadratic programming - Vanderbei - 1994
122	Support Vector Machines, Reproducing Kernel Hilbert Spaces and the randomized GACV - Wahba - 1997
122	Improving the accuracy and speed of support vector learning machines - Burges, Schölkopf - 1997
120	Simplified support vector decision rules - Burges - 1996
120	Introductory Real Analysis - Kolmogorov, Fomin - 1970
119	The connection between regularization operators and support vector kernels. Neural Networks - Smola, Schölkopf, et al. - 1998
105	Support Vector Learning - Schőlkopf - 1997
98	Support vector regression machines - Drucker, Burges, et al. - 1997
97	Comparing Support Vector Machines With Gaussian Kernels to Radial Basis Function Classifiers - Scholkopf, Sung, et al. - 1997
96	Predicting time series with support vector machines - Muller, Smola, et al. - 1997
93	Some stable methods for calculating inertia and solving symmetric linear systems - Bunch, Kaufman - 1977
91	Introduction to Linear Regression Analysis - Montgomery, Peck, et al. - 2001
79	Prior knowledge in support vector kernels - Schölkopf, Simard, et al. - 1998
67	On a kernel-based method for pattern recognition, regression, approximation and operator inversion - Smola, Scholkopf - 1998
66	A Hilbert space problem book - Halmos - 1967
63	Nonlinear prediction of chaotic time series using support vector machines - Mukherjee, Osuna, et al. - 1997
61	V.: Incorporating invariances in support vector learning machines - Schlkopf, Burges, et al. - 1996
52	On the solution of large quadratic programming problems with bound constraints - Moré, Toraldo - 1991
42	Estimation of Dependences Based on Empirical Data [in Russian - Vapnik - 1979
39	Reducing the run-time complexity in support vector machines. Advances in kernel methods: support vector learning - Osuna, Girosi - 1999
38	Nonlinear Programming: Theory, Algorithms and Applications - McCormick - 1983
33	General cost functions for support vector regression - Smola, Schölkopf, et al. - 1998
32	Structural risk minimization for character recognition', this volume - Guyon, Vapnik, et al. - 1992
28	at al. Numerical recipes in C: the art of scienti c computing Cambridge - Press - 1988
24	A framework for structural risk minimization - Shawe-Taylor, Bartlett, et al. - 1996
22	László Györfi, and Gábor Lugosi. A Probabilistic Approach to Pattern Recognition - Devroye - 1996
21	Neural networks and the bias / variance dilemma. Neural Computation - Geman, Bienstock, et al. - 1992
18	Geometry in learning - Bennett, Bredensteiner - 1998
18	Comparing support vector machines with gaussian kernels to radial basis function classifiers - Schoelkopf, Sung, et al. - 1996
16	On the solution of large quadratic programming problems with bound constraints - JJ, Toraldo - 1991
16	Identifying speaker with support vector networks - Schmidt - 1996