Results 1 
5 of
5
Distance metric learning for large margin nearest neighbor classification
 In NIPS
, 2006
"... We show how to learn a Mahanalobis distance metric for knearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the knearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven ..."
Abstract

Cited by 326 (10 self)
 Add to MetaCart
We show how to learn a Mahanalobis distance metric for knearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the knearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classificationâ€”for example, achieving a test error rate of 1.3 % on the MNIST handwritten digits. As in support vector machines (SVMs), the learning problem reduces to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our framework requires no modification or extension for problems in multiway (as opposed to binary) classification. 1
Metric Learning: A Support Vector Approach
"... Abstract. In this paper, we address the metric learning problem utilizing a marginbased approach. Our metric learning problem is formulated as a quadratic semidefinite programming problem (QSDP) with local neighborhood constraints, which is based on the Support Vector Machine (SVM) framework. The ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Abstract. In this paper, we address the metric learning problem utilizing a marginbased approach. Our metric learning problem is formulated as a quadratic semidefinite programming problem (QSDP) with local neighborhood constraints, which is based on the Support Vector Machine (SVM) framework. The local neighborhood constraints ensure that examples of the same class are separated from examples of different classes by a margin. In addition to providing an efficient algorithm to solve the metric learning problem, extensive experiments on various data sets show that our algorithm is able to produce a new distance metric to improve the performance of the classical Knearest neighbor (KNN) algorithm on the classification task. Our performance is always competitive and often significantly better than other stateoftheart metric learning algorithms. Key words: metric learning, Knearest neighbor classification, SVM 1
Improving kNearest Neighbour Classification with Distance Functions Based on Receiver Operating Characteristics
"... Abstract. The knearest neighbour (kNN) technique, due to its interpretable nature, is a simple and very intuitively appealing method to address classification problems. However, choosing an appropriate distance function for kNN can be challenging and an inferior choice can make the classifier hig ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. The knearest neighbour (kNN) technique, due to its interpretable nature, is a simple and very intuitively appealing method to address classification problems. However, choosing an appropriate distance function for kNN can be challenging and an inferior choice can make the classifier highly vulnerable to noise in the data. In this paper, we propose a new method for determining a good distance function for kNN. Our method is based on consideration of the area under the Receiver Operating Characteristics (ROC) curve, which is a well known method to measure the quality of binary classifiers. It computes weights for the distance function, based on ROC properties within an appropriate neighbourhood for the instances whose distance is being computed. We experimentally compare the effect of our scheme with a number of other wellknown kNN distance metrics, as well as with a range of different classifiers. Experiments show that our method can substantially boost the classification performance of the kNN algorithm. Furthermore, in a number of cases our technique is even able to deliver better accuracy than stateoftheart non kNN classifiers, such as support vector machines.
Abstract Semisupervised Learning of a Markovian Metric
"... The role of a distance metric in many supervised and semisupervised learning applications is central in the success of clustering algorithms. Since existing metrics like Euclidean do not necessarily reflect the true structure (clusters or manifolds) in the data, it becomes imperative that an approp ..."
Abstract
 Add to MetaCart
The role of a distance metric in many supervised and semisupervised learning applications is central in the success of clustering algorithms. Since existing metrics like Euclidean do not necessarily reflect the true structure (clusters or manifolds) in the data, it becomes imperative that an appropriate metric be somehow learned from training or labeled data. Metric learning has been a relatively new topic in data mining and machine learning, though most work that deals with this topic learns a suitable linear transformation of the original data. This transformation is usually learned using training data and has been shown to improve test data classification accuracy. In this paper we present a Markov random walk based semisupervised method for metric learning. Our method differs from the aforementioned techniques in that we use minimal labeled data and we do not assume any Mahalanobis type metric structure on the data. We create a computationally efficient nearest neighbor graph representation of the data and pose a semidefinite program that learns the random walk on the associated graph. This is used to generate a distance measure between all unlabeled points and the performance is compared against other important metrics using the kNN classification rule.
Graphbased Discrete Differential Geometry for Critical Instance Filtering
"... Abstract. Graph theory has been shown to provide a powerful tool for representing and tackling machine learning problems, such as clustering, semisupervised learning, and feature ranking. This paper proposes a graphbased discrete differential operator for detecting and eliminating competencecriti ..."
Abstract
 Add to MetaCart
Abstract. Graph theory has been shown to provide a powerful tool for representing and tackling machine learning problems, such as clustering, semisupervised learning, and feature ranking. This paper proposes a graphbased discrete differential operator for detecting and eliminating competencecritical instances and class label noise from a training set in order to improve classification performance. Results of extensive experiments on artificial and reallife classification problems substantiate the effectiveness of the proposed approach. 1