Results 1  10
of
23
Metric and Kernel Learning Using a Linear Transformation
"... Metric and kernel learning arise in several machine learning applications. However, most existing metric learning algorithms are limited to learning metrics over lowdimensional data, while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new ..."
Abstract

Cited by 30 (2 self)
 Add to MetaCart
(Show Context)
Metric and kernel learning arise in several machine learning applications. However, most existing metric learning algorithms are limited to learning metrics over lowdimensional data, while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points. In this paper, we study the connections between metric learning and kernel learning that arise when studying metric learning as a linear transformation learning problem. In particular, we propose a general optimization framework for learning metrics via linear transformations, and analyze in detail a special case of our framework—that of minimizing the LogDet divergence subject to linear constraints. We then propose a general regularized framework for learning a kernel matrix, and show it to be equivalent to our metric learning framework. Our theoretical connections between metric and kernel learning have two main consequences: 1) the learned kernel matrix parameterizes a linear transformation kernel function and can be applied inductively to new data points, 2) our result yields a constructive method for kernelizing most existing Mahalanobis metric learning formulations. We demonstrate our learning approach by applying it to largescale real world problems in computer vision, text mining and semisupervised kernel dimensionality reduction. Keywords: divergence metric learning, kernel learning, linear transformation, matrix divergences, logdet 1.
Measuring the Similarity between Implicit Semantic Relations from the Web
 WWW 2009 MADRID! TRACK: SEMANTIC/DATA WEB / SESSION: MINING FOR SEMANTICS
, 2009
"... Measuring the similarity between semantic relations that hold among entities is an important and necessary step in various Web related tasks such as relation extraction, information retrieval and analogy detection. For example, consider the case in which a person knows a pair of entities (e.g. Googl ..."
Abstract

Cited by 18 (8 self)
 Add to MetaCart
Measuring the similarity between semantic relations that hold among entities is an important and necessary step in various Web related tasks such as relation extraction, information retrieval and analogy detection. For example, consider the case in which a person knows a pair of entities (e.g. Google, YouTube), between which a particular relation holds (e.g. acquisition). The person is interested in retrieving other such pairs with similar relations (e.g. Microsoft, Powerset). Existing keywordbased search engines cannot be applied directly in this case because, in keywordbased search, the goal is to retrieve documents that are relevant to the words used in a query – not necessarily to the relations implied by a pair of words. We propose a relational similarity measure, using a Web search engine, to compute the similarity between semantic relations implied by two pairs of words. Our method has three components: representing
Transfer Metric Learning by Learning Task Relationships
"... Distance metric learning plays a very crucial role in many data mining algorithms because the performance of an algorithm relies heavily on choosing a good metric. However, the labeled data available in many applications is scarce and hence the metrics learned are often unsatisfactory. In this paper ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
(Show Context)
Distance metric learning plays a very crucial role in many data mining algorithms because the performance of an algorithm relies heavily on choosing a good metric. However, the labeled data available in many applications is scarce and hence the metrics learned are often unsatisfactory. In this paper, we consider a transfer learning setting in which some related source tasks with labeled data are available to help the learning of the target task. We first propose a convex formulation for multitask metric learning by modeling the task relationships in the form of a task covariance matrix. Then we regard transfer learning as a special case of multitask learning and adapt the formulation of multitask metric learning to the transfer learning setting for our method, called transfer metric learning (TML). In TML, we learn the metric and the task covariances between the source tasks and the target task under a unified convex formulation. To solve the convex optimization problem, we use an alternating method in which each subproblem has an efficient solution. Experimental results on some commonly used transfer learning applications demonstrate the effectiveness of our method.
Learning Discriminative Projections for Text Similarity Measures
"... Traditional text similarity measures consider each term similar only to itself and do not model semantic relatedness of terms. We propose a novel discriminative training method that projects the raw term vectors into a common, lowdimensional vector space. Our approach operates by finding the optima ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
(Show Context)
Traditional text similarity measures consider each term similar only to itself and do not model semantic relatedness of terms. We propose a novel discriminative training method that projects the raw term vectors into a common, lowdimensional vector space. Our approach operates by finding the optimal matrix to minimize the loss of the preselected similarity function (e.g., cosine) of the projected vectors, and is able to efficiently handle a large number of training examples in the highdimensional space. Evaluated on two very different tasks, crosslingual document retrieval and ad relevance measure, our method not only outperforms existing stateoftheart approaches, but also achieves high accuracy at low dimensions and is thus more efficient. 1
A Simple Gradientbased Metric Learning Algorithm for Object Recognition
"... The Nearest Neighbor (NN) classification/regression techniques, besides their simplicity, is one of the most widely applied and well studied techniques for pattern recognition in machine learning. Their only drawback is the assumption of the availability of a proper metric used to measure distances ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
The Nearest Neighbor (NN) classification/regression techniques, besides their simplicity, is one of the most widely applied and well studied techniques for pattern recognition in machine learning. Their only drawback is the assumption of the availability of a proper metric used to measure distances to k nearest neighbors. It has been shown that KNN classifier’s with a right distance metric can perform better than other sophisticated alternatives like Support Vector Machines (SVM) and Gaussian Processes (GP) classifiers. That’s why recent research in kNN methods has focused on metric learning i.e., finding an optimized metric. In this paper we have proposed a simple gradient based algorithm for metric learning. We discuss in detail the motivations behind metric learning, i.e., error minimization and margin maximization. Our formulation is different from the prevalent techniques in metric learning where goal is to maximize the classifier’s margin. Instead our proposed technique (MEGM) finds an optimal metric by directly minimizing the mean square error. Our technique not only resulted in greatly improving kNN performance but also performed better than competing metric learning techniques. We also compared our algorithm’s performance with that of SVM. Promising results are reported on major faces, digits, object and UCIML databases. 1
Metric Learning with TwoDimensional Smoothness for Visual Analysis
"... In recent years, metric learning methods based on pairwise side information have attracted considerable interests, and lots of efforts have been devoted to utilize these methods for visual analysis like content based image retrieval and face identification. When applied to image analysis, these meth ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
In recent years, metric learning methods based on pairwise side information have attracted considerable interests, and lots of efforts have been devoted to utilize these methods for visual analysis like content based image retrieval and face identification. When applied to image analysis, these methods merely look on an n1 × n2 image as a vector in R n1×n2 space and the pixels of the image are considered as independent. They fail to consider the fact that an image represented in the plane is intrinsically a matrix, and pixels spatially close to each other may probably be correlated. Even though we have n1 × n2 pixels per image, this spatial correlation suggests the real number of freedom is far less. In this paper, we introduce a regularized metric learning framework, TwoDimensional Smooth Metric Learning (2DSML), which uses a discretized Laplacian penalty to restrict the coefficients to be twodimensional smooth. Many existing metric learning algorithms can fit into this framework and learn a spatially smooth metric which is better for image applications than their original version. Recognition, clustering and retrieval can be then performed based on the learned metric. Experimental results on benchmark image datasets demonstrate the effectiveness of our method. 1.
Metric learning for reinforcement learning agents
 In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS
, 2011
"... A key component of any reinforcement learning algorithm is the underlying representation used by the agent. While reinforcement learning (RL) agents have typically relied on handcoded state representations, there has been a growing interest in learning this representation. While inputs to an agen ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
A key component of any reinforcement learning algorithm is the underlying representation used by the agent. While reinforcement learning (RL) agents have typically relied on handcoded state representations, there has been a growing interest in learning this representation. While inputs to an agent are typically fixed (i.e., state variables represent sensors on a robot), it is desirable to automatically determine the optimal relative scaling of such inputs, as well as to diminish the impact of irrelevant features. This work introduces HOLLER, a novel distance metric learning algorithm, and combines it with an existing instancebased RL algorithm to achieve precisely these goals. The algorithms ’ success is highlighted via empirical measurements on a set of six tasks within the mountain car domain.
SVMs and Data Dependent Distance Metric
"... Support Vector Machine (SVM) is an efficient classification tool. Based on the principle of structured risk minimization, SVM is designed to generalize well. But it has been shown that SVM is not immune to the curse of dimensionality. Also SVM performance is not only critical to the choice of kernel ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Support Vector Machine (SVM) is an efficient classification tool. Based on the principle of structured risk minimization, SVM is designed to generalize well. But it has been shown that SVM is not immune to the curse of dimensionality. Also SVM performance is not only critical to the choice of kernel but also to the kernel parameters which are generally tuned through computationally expensive crossvalidation procedures. Typical kernels do not have any information about the subspace to ignore irrelevant features or making relevant features explicit. Recently, a lot of progress has been made for learning a data dependent distance metric for improving the efficiency of kNearest Neighbor (KNN) classifier. Metric learning approaches have not been investigated in the context of SVM. In this paper, we study the impact of learning a data dependent distance metric on classification performance of an SVM classifier. Our novel approach in this paper is a formulation relying on a simple Mean Square Error (MSE) gradient based metric learning method to tune kernel’s parameters. Experiments are conducted on major UCIML, faces and digit databases. We have found that tuning kernel parameters through a metric learning approach can improve the classification performance of an SVM classifier.
From subspace learning to distance learning: a geometrical optimization approach
 Proceedings of the IEEE/SP 15th Workshop on Statistical Signal Processing. IEEE
, 2009
"... In this paper, we adopt a differentialgeometry viewpoint to tackle the problem of learning a distance online. As this problem can be cast into the estimation of a fixedrank positive semidefinite (PSD) matrix, we develop algorithms that exploits the rich geometry structure of the set of fixedran ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
In this paper, we adopt a differentialgeometry viewpoint to tackle the problem of learning a distance online. As this problem can be cast into the estimation of a fixedrank positive semidefinite (PSD) matrix, we develop algorithms that exploits the rich geometry structure of the set of fixedrank PSD matrices. We propose a method which separately updates the subspace of the matrix and its projection onto that subspace. A proper weighting of the two iterations enables to continuously interpolate between the problem of learning a subspace and learning a distance when the subspace is fixed. Index Terms — Kernel and metric learning, lowrank approximation, online learning, manifoldbased optimization. 1.
BoostML: An adaptive metric learning for nearest neighbor classification
 in PAKDD
, 2010
"... Abstract. The nearest neighbor classification/regression technique, besides its simplicity, is one of the most widely applied and well studied techniques for pattern recognition in machine learning. A nearest neighbor classifier assumes class conditional probabilities to be locally smooth. This as ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract. The nearest neighbor classification/regression technique, besides its simplicity, is one of the most widely applied and well studied techniques for pattern recognition in machine learning. A nearest neighbor classifier assumes class conditional probabilities to be locally smooth. This assumption is often invalid in high dimensions and significant bias can be introduced when using the nearest neighbor rule. This effect can be mitigated to some extent by using a locally adaptive metric. In this work we present a detailed analysis of the introduction of bias in high dimensional machine learning data and propose an adaptive metric learning algorithm that learns an optimal metric at the query point using respective class distributions on the input measurement space. We learn a distance metric using a feature relevance measure inspired by boosting. The modified metric results in a smooth neighborhood that leads to better classification results. We tested our technique on major UCI machine learning databases and compared the results to state of the art techniques. Our method resulted in significant improvements in the performance of the KNN classifier and also performed better than other techniques on major databases.