Results 1 
7 of
7
Feature space interpretation of svms with indefinite kernels
 IEEE Trans Pattern Anal Mach Intell
, 2005
"... Abstract—Kernel methods are becoming increasingly popular for various kinds of machine learning tasks, the most famous being the support vector machine (SVM) for classification. The SVM is well understood when using conditionally positive definite (cpd) kernel functions. However, in practice, noncp ..."
Abstract

Cited by 92 (3 self)
 Add to MetaCart
(Show Context)
Abstract—Kernel methods are becoming increasingly popular for various kinds of machine learning tasks, the most famous being the support vector machine (SVM) for classification. The SVM is well understood when using conditionally positive definite (cpd) kernel functions. However, in practice, noncpd kernels arise and demand application in SVMs. The procedure of “plugging ” these indefinite kernels in SVMs often yields good empirical classification results. However, they are hard to interpret due to missing geometrical and theoretical understanding. In this paper, we provide a step toward the comprehension of SVM classifiers in these situations. We give a geometric interpretation of SVMs with indefinite kernel functions. We show that such SVMs are optimal hyperplane classifiers not by margin maximization, but by minimization of distances between convex hulls in pseudoEuclidean spaces. By this, we obtain a sound framework and motivation for indefinite SVMs. This interpretation is the basis for further theoretical analysis, e.g., investigating uniqueness, and for the derivation of practical guidelines like characterizing the suitability of indefinite SVMs. Index Terms—Support vector machine, indefinite kernel, pseudoEuclidean space, separation of convex hulls, pattern recognition. æ 1
Training data selection for support vector machines
 ICNC 2005. LNCS
, 2005
"... In recent years, support vector machines (SVMs) have become a popular tool for pattern recognition and machine learning. Training a SVM involves solving a constrained quadratic programming problem, which requires large memory and enormous amounts of training time for largescale problems. In contra ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
In recent years, support vector machines (SVMs) have become a popular tool for pattern recognition and machine learning. Training a SVM involves solving a constrained quadratic programming problem, which requires large memory and enormous amounts of training time for largescale problems. In contrast, the SVM decision function is fully determined by a small subset of the training data, called support vectors. Therefore, it is desirable to remove from the training set the data that is irrelevant to the final decision function. In this paper we propose two new methods that select a subset of data for SVM training. Using realworld datasets, we compare the effectiveness of the proposed data selection strategies in terms of their ability to reduce the training set size while maintaining the generalization performance of the resulting SVM classifiers. Our experimental results show that a significant amount of training data can be removed by our proposed methods without degrading the performance of the resulting SVM classifiers.
Fast Support Vector Machine Classification of very large Datasets
 University of Freiburg, Department of Computer
"... Abstract. In many classification applications, Support Vector Machines (SVMs) have proven to be highly performing and easy to handle classifiers with very good generalization abilities. However, one drawback of the SVM is its rather high classification complexity which scales linearly with the numbe ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
Abstract. In many classification applications, Support Vector Machines (SVMs) have proven to be highly performing and easy to handle classifiers with very good generalization abilities. However, one drawback of the SVM is its rather high classification complexity which scales linearly with the number of Support Vectors (SVs). This is due to the fact that for the classification of one sample, the kernel function has to be evaluated for all SVs. To speed up classification, different approaches have been published, most which of try to reduce the number of SVs. In our work, which is especially suitable for very large datasets, we follow a different approach: as we showed in [12], it is effectively possible to approximate large SVM problems by decomposing the original problem into linear subproblems, where each subproblem can be evaluated in Ω(1). This approach is especially successful, when the assumption holds that a large classification problem can be split into mainly easy and only a few hard subproblems. On standard benchmark datasets, this approach achieved great speedups while suffering only sightly in terms of classification accuracy and generalization ability. In this contribution, we extend the methods introduced in [12] using not only linear, but also nonlinear subproblems for the decomposition of the original problem which further increases the classification performance with only a little loss in terms of speed. An implementation of our method is available in [13]. Due to page limitations, we had to move some of theoretic details (e.g. proofs) and extensive experimental results to a technical report [14]. 1
Selecting Data for Fast Support Vector Machine Training
 STUDIES IN COMPUTATIONAL INTELLIGENCE
, 2007
"... In recent years, support vector machines (SVMs) have become a popular tool for pattern recognition and machine learning. Training a SVM involves solving a constrained quadratic programming problem, which requires large memory and enormous amounts of training time for largescale problems. In contrast ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
In recent years, support vector machines (SVMs) have become a popular tool for pattern recognition and machine learning. Training a SVM involves solving a constrained quadratic programming problem, which requires large memory and enormous amounts of training time for largescale problems. In contrast, the SVM decision function is fully determined by a small subset of the training data, called support vectors. Therefore, it is desirable to remove from the training set the data that is irrelevant to the final decision function. In this paper we propose two new methods that select a subset of data for SVM training. Using realworld datasets, we compare the effectiveness of the proposed data selection strategies in terms of their ability to reduce the training set size while maintaining the generalization performance of the resulting SVM classifiers. Our experimental results show that a significant amount of training data can be removed by our proposed methods without degrading the performance of the resulting SVM classifiers.
A novel FrankWolfe algorithm. analysis and applications to largescale SVM training. Information Sciences (in press
, 2014
"... Recently, there has been a renewed interest in the machine learning community for variants of a sparse greedy approximation procedure for concave optimization known as the FrankWolfe (FW) method. In particular, this procedure has been successfully applied to train largescale instances of nonline ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Recently, there has been a renewed interest in the machine learning community for variants of a sparse greedy approximation procedure for concave optimization known as the FrankWolfe (FW) method. In particular, this procedure has been successfully applied to train largescale instances of nonlinear Support Vector Machines (SVMs). Specializing FW to SVM training has allowed to obtain efficient algorithms but also important theoretical results, including convergence analysis of training algorithms and new characterizations of model sparsity. In this paper, we present and analyze a novel variant of the FWmethod based on a new way to perform away steps, a classic strategy used to accelerate the convergence of the basic FW procedure. Our formulation and analysis is focused on a general concave maximization problem on the simplex. However, the specialization of our algorithm to quadratic forms is strongly related to some classic methods in computational geometry, namely the Gilbert and MDM algorithms. On the theoretical side, we demonstrate that the method matches the guarantees in terms of convergence rate and number of iterations obtained by using classic away steps. In particular, the method enjoys a linear rate of convergence, a result that has been recently proved for MDM on quadratic forms. On the practical side, we provide experiments on several classification datasets, and evaluate the results using statistical tests. Experiments show that our method is faster than the FW method with classic away steps, and works well even in the cases in which classic away steps slow down the algorithm. Furthermore, these improvements are obtained without sacrificing the predictive accuracy of the obtained SVM model. 1 ar
Feature Space Interpretation of SVMs with non Positive Definite Kernels
, 2003
"... The widespread habit of “plugging ” arbitrary symmetric functions as kernels in support vector machines (SVMs) often yields good empirical classification results. However, in case of non conditionally positive definite (noncpd) functions they are hard to interpret due to missing geometrical and the ..."
Abstract
 Add to MetaCart
(Show Context)
The widespread habit of “plugging ” arbitrary symmetric functions as kernels in support vector machines (SVMs) often yields good empirical classification results. However, in case of non conditionally positive definite (noncpd) functions they are hard to interpret due to missing geometrical and theoretical understanding. In this paper we provide a step towards comprehension of SVM classifiers in these situations. We give a geometric interpretation of SVMs with noncpd kernel functions. We show that such SVMs are optimal hyperplane classifiers not by margin maximization but by minimization of distances between convex hulls in pseudoEuclidean spaces. This interpretation is basis for further analysis, e.g. investigating uniqueness or characterizing situations where SVMs with noncpd kernels are suitable or not.
Statistical Signal Processing for Novelty Detection.
"... The goal of this article is to investigate and suggest techniques for health condition monitoring and diagnosis using machine learning from sensor data. In particular, this article overview and discusses support vector machines methods such as hard margin and soft margin problems. In order to in ..."
Abstract
 Add to MetaCart
The goal of this article is to investigate and suggest techniques for health condition monitoring and diagnosis using machine learning from sensor data. In particular, this article overview and discusses support vector machines methods such as hard margin and soft margin problems. In order to investigate the abnormalities and classify a large set of data an iterative Support Vector Machine algorithm was constructed. However, similar techniques could be applied to analyze or monitor for abnormality various other complex devices or even computer methods. Key words Support Vector Machines, Health condition monitoring, Novelty detection and Machine learning methods.