Results 1  10
of
29
An introduction to boosting and leveraging
 Advanced Lectures on Machine Learning, LNCS
, 2003
"... ..."
Generalized Kernel Approach to Dissimilaritybased Classification
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2001
"... Usually, objects to be classified are represented by features. In this paper, we discuss an alternative object representation based on dissimilarity values. If such distances separate the classes well, the nearest neighbor method offers a good solution. However, dissimilarities used in practice are ..."
Abstract

Cited by 53 (2 self)
 Add to MetaCart
Usually, objects to be classified are represented by features. In this paper, we discuss an alternative object representation based on dissimilarity values. If such distances separate the classes well, the nearest neighbor method offers a good solution. However, dissimilarities used in practice are usually far from ideal and the performance of the nearest neighbor rule suffers from its sensitivity to noisy examples. We show that other, more global classification techniques are preferable to the nearest neighbor rule, in such cases. For classification purposes, two different ways of using generalized dissimilarity kernels are considered. In the first one, distances are isometrically embedded in a pseudoEuclidean space and the classification task is performed there. In the second approach, classifiers are built directly on distance kernels. Both approaches are described theoretically and then compared using experiments with different dissimilarity measures and datasets including degraded data simulating the problem of missing values.
Prototype selection for dissimilaritybased classifiers
 Pattern Recognition
, 2006
"... A conventional way to discriminate between objects represented by dissimilarities is the nearest neighbor method. A more efficient and sometimes a more accurate solution is offered by other dissimilaritybased classifiers. They construct a decision rule based on the entire training set, but they nee ..."
Abstract

Cited by 34 (2 self)
 Add to MetaCart
A conventional way to discriminate between objects represented by dissimilarities is the nearest neighbor method. A more efficient and sometimes a more accurate solution is offered by other dissimilaritybased classifiers. They construct a decision rule based on the entire training set, but they need just a small set of prototypes, the socalled representation set, as a reference for classifying new objects. Such alternative approaches may be especially advantageous for nonEuclidean or even nonmetric dissimilarities. The choice of a proper representation set for dissimilaritybased classifiers is not yet fully investigated. It appears that a random selection may work well. In this paper, a number of experiments has been conducted on various metric and nonmetric dissimilarity representations and prototype selection methods. Several procedures, like traditional feature selection methods (here effectively searching for prototypes), mode seeking and linear programming are compared to the random selection. In general, we find out that systematic approaches lead to better results than the random selection, especially for a small number of prototypes. Although there is no single winner as it depends on data characteristics, the kcentres works well, in general. For twoclass problems, an important observation is that our dissimilaritybased discrimination functions relying on significantly reduced prototype sets (3–10 % of the training objects) offer a similar or much better classification accuracy than the best kNN rule on the entire training set. This may be reached for multiclass data as well, however such problems are more difficult.
2004), Learning with distance substitution kernels
 in Pattern Rcognition  Proc. of the 26th DAGM Symposium
"... Abstract. During recent years much effort has been spent in incorporating problem specific apriori knowledge into kernel methods for machine learning. A common example is apriori knowledge given by a distance measure between objects. A simple but effective approach for kernel construction consists ..."
Abstract

Cited by 26 (2 self)
 Add to MetaCart
Abstract. During recent years much effort has been spent in incorporating problem specific apriori knowledge into kernel methods for machine learning. A common example is apriori knowledge given by a distance measure between objects. A simple but effective approach for kernel construction consists of substituting the Euclidean distance in ordinary kernel functions by the problem specific distance measure. We formalize this distance substitution procedure and investigate theoretical and empirical effects. In particular we state criteria for definiteness of the resulting kernels. We demonstrate the wide applicability by solving several classification tasks with SVMs. Regularization of the kernel matrices can additionally increase the recognition accuracy. 1
Similaritybased Classification: Concepts and Algorithms
, 2008
"... This report reviews and extends the field of similaritybased classification, presenting new analyses, algorithms, data sets, and the most comprehensive set of experimental results to date. Specifically, the generalizability of using similarities as features is analyzed, design goals and methods for ..."
Abstract

Cited by 24 (2 self)
 Add to MetaCart
This report reviews and extends the field of similaritybased classification, presenting new analyses, algorithms, data sets, and the most comprehensive set of experimental results to date. Specifically, the generalizability of using similarities as features is analyzed, design goals and methods for weighting nearestneighbors for similaritybased learning are proposed, and different methods for consistently converting similarities into kernels are compared. Experiments on eight real data sets compare eight approaches and their variants to similaritybased learning. 1
Local Regularization Assisted Orthogonal Least Squares Regression
 IEEE Transactions on Neural Networks, submitted
, 2001
"... A locally regularized orthogonal least squares (LROLS) algorithm is proposed for constructing parsimonious or sparse regression models that generalize well. By associating each orthogonal weight in the regression model with an individual regularization parameter, the ability for the orthogonal least ..."
Abstract

Cited by 23 (5 self)
 Add to MetaCart
A locally regularized orthogonal least squares (LROLS) algorithm is proposed for constructing parsimonious or sparse regression models that generalize well. By associating each orthogonal weight in the regression model with an individual regularization parameter, the ability for the orthogonal least squares (OLS) model selection to produce a very sparse model with good generalization performance is greatly enhanced. Furthermore, with the assistance of local regularization, when to terminate the model selection procedure becomes much clearer. This LROLS algorithm has computational advantages over the recently introduced relevance vector machine (RVM) method. Keywords Orthogonal least squares algorithm, regularization, regression, support vector machines, relevance vector machines. I.
Distancebased classification with lipschitz functions
 Journal of Machine Learning Research
, 2003
"... The goal of this article is to develop a framework for large margin classification in metric spaces. We want to find a generalization of linear decision functions for metric spaces and define a corresponding notion of margin such that the decision function separates the training points with a large ..."
Abstract

Cited by 21 (2 self)
 Add to MetaCart
The goal of this article is to develop a framework for large margin classification in metric spaces. We want to find a generalization of linear decision functions for metric spaces and define a corresponding notion of margin such that the decision function separates the training points with a large margin. It will turn out that using Lipschitz functions as decision functions, the inverse of the Lipschitz constant can be interpreted as the size of a margin. In order to construct a clean mathematical setup we isometrically embed the given metric space into a Banach space and the space of Lipschitz functions into its dual space. To analyze the resulting algorithm, we prove several representer theorems. They state that there always exist solutions of the Lipschitz classifier which can be expressed in terms of distance functions to training points. We provide generalization bounds for Lipschitz classifiers in terms of the Rademacher complexities of some Lipschitz function classes. The generality of our approach can be seen from the fact that several wellknown algorithms are special cases of the Lipschitz classifier, among them the support vector machine, the linear programming machine, and the 1nearest neighbor classifier. 1.
A Generalized Kernel Approach to Dissimilarity Based Classification
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2002
"... Usually, objects to be classified are represented by features. In this paper, we discuss an alternative object representation based on dissimilarity values. If such distances separate the classes well, the nearest neighbor method offers a good solution. However, dissimilarities used in practice are ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
Usually, objects to be classified are represented by features. In this paper, we discuss an alternative object representation based on dissimilarity values. If such distances separate the classes well, the nearest neighbor method offers a good solution. However, dissimilarities used in practice are usually far from ideal and the performance of the nearest neighbor rule suffers from its sensitivity to noisy examples. We show that other, more global classification techniques are preferable to the nearest neighbor rule, in such cases. For classification purposes, two different ways of using generalized dissimilarity kernels are considered. In the first one, distances are isometrically embedded in a pseudoEuclidean space and the classification task is performed there. In the second approach, classifiers are built directly on distance kernels. Both approaches are described theoretically and then compared using experiments with different dissimilarity measures and datasets including degraded data simulating the problem of missing values.
Combining sparsity and rotational invariance in EEG/MEG source reconstruction
, 2008
"... We introduce Focal Vector Field Reconstruction (FVR), a novel technique for the inverse imaging of vector fields. The method was designed to simultaneously achieve two goals: a) invariance with respect to the orientation of the coordinate system, and b) a preference for sparsity of the solutions and ..."
Abstract

Cited by 15 (9 self)
 Add to MetaCart
We introduce Focal Vector Field Reconstruction (FVR), a novel technique for the inverse imaging of vector fields. The method was designed to simultaneously achieve two goals: a) invariance with respect to the orientation of the coordinate system, and b) a preference for sparsity of the solutions and their spatial derivatives. This was achieved by defining the regulating penalty function, which renders the solutions unique, as a global ℓ1norm of local ℓ2norms. We show that the method can be successfully used for solving the EEG inverse problem. In the joint localization of 23 simulated dipoles, FVR always reliably recovers the true sources. The competing methods have limitations in distinguishing close sources because their estimates are either too smooth (LORETA, Minimum ℓ2norm) or too scattered (Minimum ℓ1norm). In both noiseless and noisy simulations, FVR has the smallest localization error according to the Earth Mover’s Distance (EMD), which is introduced here as a meaningful measure to compare arbitrary source distributions. We also apply the method to the simultaneous localization of left and right somatosensory N20 generators from real EEG recordings. Compared to its peers FVR was the only method that delivered correct location of the source in the somatosensory area of each hemisphere in accordance with neurophysiological prior knowledge.