## An efficient algorithm for local distance metric learning (2006)

### Cached

### Download Links

- [web.cse.msu.edu]
- [www.cs.cmu.edu]
- [www.cs.cmu.edu]
- [www.aaai.org]
- [www.aaai.org]
- [www.lb.cs.cmu.edu]
- [www.cs.cmu.edu]
- [www-2.cs.cmu.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | in Proceedings of AAAI |

Citations: | 31 - 9 self |

### BibTeX

@INPROCEEDINGS{Yang06anefficient,

author = {Liu Yang and Rong Jin},

title = {An efficient algorithm for local distance metric learning},

booktitle = {in Proceedings of AAAI},

year = {2006}

}

### OpenURL

### Abstract

Learning application-specific distance metrics from labeled data is critical for both statistical classification and information retrieval. Most of the earlier work in this area has focused on finding metrics that simultaneously optimize compactness and separability in a global sense. Specifically, such distance metrics attempt to keep all of the data points in each class close together while ensuring that data points from different classes are separated. However, particularly when classes exhibit multimodal data distributions, these goals conflict and thus cannot be simultaneously satisfied. This paper proposes a Local Distance Metric (LDM) that aims to optimize local compactness and local separability. We present an efficient algorithm that employs eigenvector analysis and bound optimization to learn the LDM from training data in a probabilistic framework. We demonstrate that LDM achieves significant improvements in both classification and retrieval accuracy compared to global distance learning and kernel-based KNN.

### Citations

9021 | The Nature of Statistical Learning Theory
- Vapnik
- 1995
(Show Context)
Citation Context ...′ � � ) = exp . Parameter λ is determined − �x−x′ � 2 2 λ by a cross validation using a 20/80 split of the training data. The second baseline is the Support Vector Machine (SVM) using the RBF kernel (=-=Vapnik 1998-=-). The third baseline approach is based on the global distance metric learning algorithm (Xing et al. 2003). Specifically, we assume a logistic regression model for estimating the probability that two... |

3703 |
Convex optimization
- Boyd, Vandenberghe
- 2004
(Show Context)
Citation Context ... Algorithm for local distance metric learning Note that the objective function Q({γi} K i=1 ) is a concave function in terms of all γs because of the convexity of the log-sum-of-exponential function (=-=Boyd & Vandenberghe 2004-=-). Hence, the optimal solution that maximizes Q({γi} K i=1 ) can be efficiently obtained using Newton’s method. We refer to this algorithm as “Local Distance Metric Learning”, or LDM for short. Figure... |

1696 | A global geometric framework for nonlinear dimensionality reduction, Science 290 - Tenenbaum, Silva, et al. - 2000 |

740 | Laplacian eigenmaps for dimensionality reduction and data representation
- Belkin, Niyogi
- 2003
(Show Context)
Citation Context ...observed data points are preserved. Popular algorithms in this category include ISOMAP (Tenenbaum, de Silva, & Langford 2000), Local Linear Embedding (Saul & Roweis 2003), and the Laplacian Eigenmap (=-=Belkin & Niyogi 2003-=-). Copyright c○ 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Rahul Sukthankar Intel Research & Carnegie Mellon 4720 Forbes Avenue, Suite 410 Pittsburgh, ... |

510 | Distance metric learning with application to clustering with side-information
- Xing, Ng, et al.
- 2002
(Show Context)
Citation Context ...mpt to learn metrics that keep data points within the same classes close, while separating data points from different classes. Examples include (Hastie & Tibshirani 1996; Domeniconi & Gunopulos 2002; =-=Xing et al. 2003-=-; Zhang, Tang, & Kwok 2005; Goldberger et al. 2005; Weinberger, Blitzer, & Saul 2006; Shalev-Shwartz, Singer, & Ng 2004). This paper focuses on learning distance metrics in a supervised setting. Most ... |

492 | An evaluation of statistical approaches to text categorization
- Yang
- 1999
(Show Context)
Citation Context ...h image is represented by 36 different visual features that belong to three categories: color, edge, and texture. • Text Categorization. We randomly select five categories from the Newsgroup dataset (=-=Yang 1999-=-) for text categorization and randomly select 100 documents for each category, resulting in 500 documents. The mutual information (Yang 1999) is used to identify the top 100 most informative words for... |

333 | Distance Metric Learning for Large Margin Nearest Neighbor Classification - Weinberger, Blitzer, et al. - 2005 |

255 | Think globally, fit locally: unsupervised learning of low dimensional manifolds
- Saul, Roweis
(Show Context)
Citation Context ...lationships (e.g., distance) between most of the observed data points are preserved. Popular algorithms in this category include ISOMAP (Tenenbaum, de Silva, & Langford 2000), Local Linear Embedding (=-=Saul & Roweis 2003-=-), and the Laplacian Eigenmap (Belkin & Niyogi 2003). Copyright c○ 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Rahul Sukthankar Intel Research & Carnegi... |

244 | Discriminant adaptive nearest neighbor classification
- Hastie, Tibshirani
- 1996
(Show Context)
Citation Context ... metric learning. Most approaches in this category attempt to learn metrics that keep data points within the same classes close, while separating data points from different classes. Examples include (=-=Hastie & Tibshirani 1996-=-; Domeniconi & Gunopulos 2002; Xing et al. 2003; Zhang, Tang, & Kwok 2005; Goldberger et al. 2005; Weinberger, Blitzer, & Saul 2006; Shalev-Shwartz, Singer, & Ng 2004). This paper focuses on learning ... |

211 | Neighbourhood Component Analysis
- Goldberger, Roweis, et al.
- 1989
(Show Context)
Citation Context ...stance learning and kernel-based KNN. Introduction Distance metric learning has played a significant role in both statistical classification and information retrieval. For instance, previous studies (=-=Goldberger et al. 2005-=-; Weinberger, Blitzer, & Saul 2006) have shown that appropriate distance metrics can significantly improve the classification accuracy of the K Nearest Neighbor (KNN) algorithm. In multimedia informat... |

80 |
Text categorization with suport vector machines: Learning with many relevant features
- Joachims
- 1998
(Show Context)
Citation Context ...r the text data. cantly better classification accuracy than the other two distance metrics, and a similar classification accuracy as SVM (which is regarded as the best method for text classification (=-=Joachims 1998-=-)). Figure 5 presents the text retrieval accuracy of the three distance metrics for the top 20 ranks. Again, we observe that the retrieval accuracy of the local distance metric is significantly better... |

65 | Manifold ranking based image retrieval - He, Li, et al. - 2004 |

60 | Learning a Semantic Space from User’s Relevance Feedback for Image Retrieval
- He, King, et al.
- 2003
(Show Context)
Citation Context ...006) have shown that appropriate distance metrics can significantly improve the classification accuracy of the K Nearest Neighbor (KNN) algorithm. In multimedia information retrieval, several papers (=-=He et al. 2003-=-; 2004; Muller, Pun, & Squire 2004) have shown that appropriate distance metrics, learned either from labeled or unlabeled data, usually result in substantial improvements in retrieval accuracy compar... |

55 | Online and Batch Learning of Pseudo-Metrics - Shalev-Shwartz, Singer, et al. - 2004 |

54 |
Learning with idealized kernels
- Kwok, Tsang
- 2003
(Show Context)
Citation Context ...ata pairs in the equivalence constraints subject to the constraint that the data pairs in the ineqivalence constraints are well separated. This algorithm is further extended to the nonlinear case in (=-=Kwok & Tsang 2003-=-) by the introduction of kernels. In addition to general purpose algorithms for distance metric learning, several papers have presented approaches to learn appropriate distance metrics for the KNN cla... |

30 | Adaptive overrelaxed bound optimization methods
- Salakhutdinov, Roweis
- 2003
(Show Context)
Citation Context ...atively close together. Then, we can learn a distance metric that satisfies this subset of constraints. To accomplish this, we employ an iterative procedure based on the bound optimization algorithm (=-=Salakhutdinov & Roweis 2003-=-). Specifically, we initialize our algorithm by using the Euclidean metric to identify the initial set of local constraints. Then we alternately iterate between the step of local distance metric learn... |

28 | Adaptive nearest neighbor classification using support vector machines
- Domeniconi
- 2002
(Show Context)
Citation Context ...roaches in this category attempt to learn metrics that keep data points within the same classes close, while separating data points from different classes. Examples include (Hastie & Tibshirani 1996; =-=Domeniconi & Gunopulos 2002-=-; Xing et al. 2003; Zhang, Tang, & Kwok 2005; Goldberger et al. 2005; Weinberger, Blitzer, & Saul 2006; Shalev-Shwartz, Singer, & Ng 2004). This paper focuses on learning distance metrics in a supervi... |

22 | Learning from user behavior in image retrieval: Application of market basket analysis - Muller, Pun, et al. - 2007 |

5 | K.T.Cheng. Svm binary classifier ensembles for multiclass image classification - Goh - 2001 |