Results 1  10
of
13
Distance metric learning for large margin nearest neighbor classification
 In NIPS
, 2006
"... We show how to learn a Mahanalobis distance metric for knearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the knearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven ..."
Abstract

Cited by 688 (15 self)
 Add to MetaCart
(Show Context)
We show how to learn a Mahanalobis distance metric for knearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the knearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification—for example, achieving a test error rate of 1.3 % on the MNIST handwritten digits. As in support vector machines (SVMs), the learning problem reduces to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our framework requires no modification or extension for problems in multiway (as opposed to binary) classification. 1
IEEE,” Efficient Algorithm for Localized Support Vector Machine
 IEEE Transactions on Knowledge and Data Engineering
, 2010
"... Abstract—This paper presents a framework called Localized Support Vector Machine (LSVM) for classifying data with nonlinear decision surfaces. Instead of building a sophisticated global model from the training data, LSVM constructs multiple linear SVMs, each of which is designed to accurately classi ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
(Show Context)
Abstract—This paper presents a framework called Localized Support Vector Machine (LSVM) for classifying data with nonlinear decision surfaces. Instead of building a sophisticated global model from the training data, LSVM constructs multiple linear SVMs, each of which is designed to accurately classify a given test example. A major limitation of this framework is its high computational cost since a unique model must be constructed for each test example. To overcome this limitation, we propose an efficient implementation of LSVM, termed Profile SVM (PSVM). PSVM partitions the training examples into clusters and builds a separate linear SVM model for each cluster. Our empirical results show that (1) LSVM and PSVM outperform nonlinear SVM for all twenty of the evaluated data sets; and (2) PSVM achieves comparable performance as LSVM in terms of model accuracy but with significant computational savings. We also demonstrate the efficacy of the proposed approaches in terms of classifying data with spatial and temporal dependencies.
Metric Learning: A Support Vector Approach
"... Abstract. In this paper, we address the metric learning problem utilizing a marginbased approach. Our metric learning problem is formulated as a quadratic semidefinite programming problem (QSDP) with local neighborhood constraints, which is based on the Support Vector Machine (SVM) framework. The ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper, we address the metric learning problem utilizing a marginbased approach. Our metric learning problem is formulated as a quadratic semidefinite programming problem (QSDP) with local neighborhood constraints, which is based on the Support Vector Machine (SVM) framework. The local neighborhood constraints ensure that examples of the same class are separated from examples of different classes by a margin. In addition to providing an efficient algorithm to solve the metric learning problem, extensive experiments on various data sets show that our algorithm is able to produce a new distance metric to improve the performance of the classical Knearest neighbor (KNN) algorithm on the classification task. Our performance is always competitive and often significantly better than other stateoftheart metric learning algorithms. Key words: metric learning, Knearest neighbor classification, SVM 1
Improving kNearest Neighbour Classification with Distance Functions Based on Receiver Operating Characteristics
"... Abstract. The knearest neighbour (kNN) technique, due to its interpretable nature, is a simple and very intuitively appealing method to address classification problems. However, choosing an appropriate distance function for kNN can be challenging and an inferior choice can make the classifier hig ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract. The knearest neighbour (kNN) technique, due to its interpretable nature, is a simple and very intuitively appealing method to address classification problems. However, choosing an appropriate distance function for kNN can be challenging and an inferior choice can make the classifier highly vulnerable to noise in the data. In this paper, we propose a new method for determining a good distance function for kNN. Our method is based on consideration of the area under the Receiver Operating Characteristics (ROC) curve, which is a well known method to measure the quality of binary classifiers. It computes weights for the distance function, based on ROC properties within an appropriate neighbourhood for the instances whose distance is being computed. We experimentally compare the effect of our scheme with a number of other wellknown kNN distance metrics, as well as with a range of different classifiers. Experiments show that our method can substantially boost the classification performance of the kNN algorithm. Furthermore, in a number of cases our technique is even able to deliver better accuracy than stateoftheart non kNN classifiers, such as support vector machines.
Abstract Semisupervised Learning of a Markovian Metric
"... The role of a distance metric in many supervised and semisupervised learning applications is central in the success of clustering algorithms. Since existing metrics like Euclidean do not necessarily reflect the true structure (clusters or manifolds) in the data, it becomes imperative that an approp ..."
Abstract
 Add to MetaCart
(Show Context)
The role of a distance metric in many supervised and semisupervised learning applications is central in the success of clustering algorithms. Since existing metrics like Euclidean do not necessarily reflect the true structure (clusters or manifolds) in the data, it becomes imperative that an appropriate metric be somehow learned from training or labeled data. Metric learning has been a relatively new topic in data mining and machine learning, though most work that deals with this topic learns a suitable linear transformation of the original data. This transformation is usually learned using training data and has been shown to improve test data classification accuracy. In this paper we present a Markov random walk based semisupervised method for metric learning. Our method differs from the aforementioned techniques in that we use minimal labeled data and we do not assume any Mahalanobis type metric structure on the data. We create a computationally efficient nearest neighbor graph representation of the data and pose a semidefinite program that learns the random walk on the associated graph. This is used to generate a distance measure between all unlabeled points and the performance is compared against other important metrics using the kNN classification rule.
Graphbased Discrete Differential Geometry for Critical Instance Filtering
"... Abstract. Graph theory has been shown to provide a powerful tool for representing and tackling machine learning problems, such as clustering, semisupervised learning, and feature ranking. This paper proposes a graphbased discrete differential operator for detecting and eliminating competencecriti ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. Graph theory has been shown to provide a powerful tool for representing and tackling machine learning problems, such as clustering, semisupervised learning, and feature ranking. This paper proposes a graphbased discrete differential operator for detecting and eliminating competencecritical instances and class label noise from a training set in order to improve classification performance. Results of extensive experiments on artificial and reallife classification problems substantiate the effectiveness of the proposed approach. 1
Dynamic time warping
"... rping ficatio e sim by a In this paper, we try to address the problem of similarity measurement of time series by adjusting the constraints of DTW. In the last decade, the large margin criterion has been widely discussed in feature evaluation, distance learning and classification modeling. Accordin ..."
Abstract
 Add to MetaCart
(Show Context)
rping ficatio e sim by a In this paper, we try to address the problem of similarity measurement of time series by adjusting the constraints of DTW. In the last decade, the large margin criterion has been widely discussed in feature evaluation, distance learning and classification modeling. According to the statistical learning theory, a classifier with large margin will produce good generalization performance. In this work, we introduce this criterion into the global constraint learning for dynamic time warping. Based on
Submitted to IEEE Trans. on Neural Networks and Learning Systems 1
"... Abstract — A key challenge in largescale image classification is how to achieve efficiency in terms of both computation and memory without compromising classification accuracy. The learningbased classifiers achieve the stateoftheart accuracies, but have been criticized for the computational com ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — A key challenge in largescale image classification is how to achieve efficiency in terms of both computation and memory without compromising classification accuracy. The learningbased classifiers achieve the stateoftheart accuracies, but have been criticized for the computational complexity that grows linearly with the number of classes. The nonparametric NearestNeighbor (NN) based classifiers naturally handle large numbers of categories, but incur prohibitively expensive computation and memory costs. In this paper, we present a novel classification scheme, i.e., Discriminative Hierarchical KMeans Tree (DHKTree), which combines the advantages of both learningbased and NNbased classifiers. The complexity of the DHKTree only grows sublinearly with the number of categories, which is much better than the recent hierarchical Support Vector Machines (SVM) based methods. The memory requirement is the order of magnitude less than the recent NaïveBayesian NearestNeighbor (NBNN) based approaches. The proposed DHKTree classification scheme is evaluated on several challenging benchmark databases and achieves the stateoftheart accuracies, while with significantly lower computation cost and memory requirement.
Automatic Adjustment of Discriminant Adaptive Nearest Neighbor
"... KNearest Neighbors relies on the definition of a global metric. In contrast, Discriminant Adaptive Nearest Neighbor (DANN) computes a different metric at each query point based on a local Linear Discriminant Analysis. In this paper, we propose a technique to automatically adjust the hyperparameter ..."
Abstract
 Add to MetaCart
(Show Context)
KNearest Neighbors relies on the definition of a global metric. In contrast, Discriminant Adaptive Nearest Neighbor (DANN) computes a different metric at each query point based on a local Linear Discriminant Analysis. In this paper, we propose a technique to automatically adjust the hyperparameters in DANN by the optimization of two quality criteria. The first one measures the quality of discrimination, while the second one maximizes the local class homogeneity. We use a Bayesian formulation to prevent overfitting. 1.