## Kernelized locality-sensitive hashing for scalable image search (2009)

### Cached

### Download Links

Venue: | IEEE International Conference on Computer Vision (ICCV |

Citations: | 72 - 2 self |

### BibTeX

@INPROCEEDINGS{Kulis09kernelizedlocality-sensitive,

author = {Brian Kulis and Kristen Grauman},

title = {Kernelized locality-sensitive hashing for scalable image search},

booktitle = {IEEE International Conference on Computer Vision (ICCV},

year = {2009}

}

### OpenURL

### Abstract

Fast retrieval methods are critical for large-scale and data-driven vision applications. Recent work has explored ways to embed high-dimensional features or complex distance functions into a low-dimensional Hamming space where items can be efficiently searched. However, existing methods do not apply for high-dimensional kernelized data when the underlying feature embedding for the kernel is unknown. We show how to generalize locality-sensitive hashing to accommodate arbitrary kernel functions, making it possible to preserve the algorithm’s sub-linear time similarity search guarantees for a wide class of useful similarity functions. Since a number of successful image-based kernels have unknown or incomputable embeddings, this is especially valuable for image retrieval tasks. We validate our technique on several large-scale datasets, and show that it enables accurate and fast performance for example-based object classification, feature matching, and content-based retrieval. 1.

### Citations

1049 | Nonlinear component analysis as a kernel eigenvalue problem
- Smola, Scholkopf, et al.
- 1996
(Show Context)
Citation Context ...mean as µ = 1 p form the covariance matrix Σ over the p samples, though we cannot store this matrix explicitly. In order to compute h(φ(x)), we will use a technique similar to that used in kernel PCA =-=[23]-=- to project onto the eigenvectors of the covariance matrix. In our case, if the eigendecomposition of Σ is V ΛV T , then Σ 1/2 = V Λ 1/2 V T . Therefore, h(φ(x)) = sign(φ(x) T V Λ 1/2 V T ˜zt). Define... |

938 | Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
- GOEMANS, WILLIAMSON
- 1995
(Show Context)
Citation Context ...n multivariate Gaussian N(0, I) of the same dimensionality as the input x. The fact that this hash function obeys the localitysensitive hash property follows from a result from Goemans and Williamson =-=[11]-=-, who showed for such a random r that Pr[sign(x T i r) = sign(xT j r)] = 1− 1 π cos−1 ( x T i xj ‖xi‖‖xj‖ Procedurally, one chooses a random vector r from N(0, I), then computes the sign of r T x for ... |

713 | Approximate nearest neighbors: towards removing the curse of dimensionality, Proceedings of the thirtieth annual ACM symposium on Theory of computing
- Indyk, Motwani
- 1998
(Show Context)
Citation Context ... high-dimensional inputs. Locality-sensitive hashing [10, 3] offers sub-linear time search by hashing highly similar examples together in a hash table; LSH functions that accommodate Hamming distance =-=[15]-=-, inner products [3], ℓp norms [6], normalized partial matching [12], and learned Mahalanobis metrics [16] have all been developed in prior work. Vision researchers have shown the effectiveness of thi... |

624 | Scalable recognition with a vocabulary tree
- Nistér, Stewénius
- 2006
(Show Context)
Citation Context ...query time guarantee. Since high-dimensional image descriptors are commonly used in object recognition, methods to mitigate these factors have been explored, such as hierarchical feature quantization =-=[18]-=-, decision trees [19], and priority queues [2]. Tree-based search structures that can operate with arbitrary metrics [29, 5] remove the assumption of a vector space by exploiting the triangle inequali... |

514 | M-tree: An efficient Access method for Similarity Search in Metric Spaces
- Ciaccia, patella, et al.
- 1997
(Show Context)
Citation Context ...te these factors have been explored, such as hierarchical feature quantization [18], decision trees [19], and priority queues [2]. Tree-based search structures that can operate with arbitrary metrics =-=[29, 5]-=- remove the assumption of a vector space by exploiting the triangle inequality. However, in practice selecting useful partitioning strategies requires good heuristics, and, in spite of logarithmic que... |

464 | Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories
- Fei-Fei, Fergus, et al.
(Show Context)
Citation Context ...re space embeddings (φ(x), φ(y)) are either unknown or incomputable. We empirically validate our scheme with several visual search tasks. For object recognition, we present results on the Caltech-101 =-=[8]-=-, and show that our hashing scheme outperforms existing hashing methods on this data set since it can compute hash functions over arbitrary kernels. For feature indexing with a larger database, we pro... |

417 | Similarity search in high dimensions via hashing
- Gionis, Indyk, et al.
- 1999
(Show Context)
Citation Context ...predictable loss in accuracy is sacrificed in order to allow fast queries even for high-dimensional inputs [25, 24, 12, 16, 4]. Methods for this problem, most notably locality-sensitive hashing (LSH) =-=[10, 3]-=-, offer probabilistic guarantees of retrieving items within (1 + ǫ) times the optimal similarity, with query times that are sub-linear with respect to n. The basic idea is to compute randomized hash f... |

376 | Local features and kernels for classification of texture and object categories: A comprehensive study
- Zhang, Marsza, et al.
- 2007
(Show Context)
Citation Context ...el function is computable). It is thus far impossible to apply LSH and its variants to search data with a number of powerful kernels—including many kernels designed specifically for image comparisons =-=[33, 34, 30]-=-, as well as some basic well-used functions like a Gaussian RBF. Further, since visual representations are often most naturally encoded with structured inputs (e.g., sets, graphs, trees), the lack of ... |

289 | Localitysensitive hashing scheme based on p-stable distributions
- Datar, Immorlica, et al.
- 2004
(Show Context)
Citation Context ...sensitive hashing [10, 3] offers sub-linear time search by hashing highly similar examples together in a hash table; LSH functions that accommodate Hamming distance [15], inner products [3], ℓp norms =-=[6]-=-, normalized partial matching [12], and learned Mahalanobis metrics [16] have all been developed in prior work. Vision researchers have shown the effectiveness of this class of methods for various ima... |

241 | 80 million tiny images: A large data set for nonparametric object and scene recognition
- Torralba, Fergus, et al.
(Show Context)
Citation Context ...nels. For feature indexing with a larger database, we provide results on the Photo Tourism data set of local patches [26, 14]. Finally, we experiment with the Tiny Image data set of 80 million images =-=[27]-=-, in order to show our technique’s ability to scale to very large databases. Because our algorithm enables fast approximate search for arbitrary kernels, we can now access a much wider class of simila... |

230 | Similarity estimation techniques from rounding algorithms
- Charikar
- 2002
(Show Context)
Citation Context ...predictable loss in accuracy is sacrificed in order to allow fast queries even for high-dimensional inputs [25, 24, 12, 16, 4]. Methods for this problem, most notably locality-sensitive hashing (LSH) =-=[10, 3]-=-, offer probabilistic guarantees of retrieving items within (1 + ǫ) times the optimal similarity, with query times that are sub-linear with respect to n. The basic idea is to compute randomized hash f... |

210 | SVM-KNN: Discriminative nearest neighbor classification for visual category recognition
- Zhang, Berg, et al.
- 2006
(Show Context)
Citation Context ...el function is computable). It is thus far impossible to apply LSH and its variants to search data with a number of powerful kernels—including many kernels designed specifically for image comparisons =-=[33, 34, 30]-=-, as well as some basic well-used functions like a Gaussian RBF. Further, since visual representations are often most naturally encoded with structured inputs (e.g., sets, graphs, trees), the lack of ... |

190 | Shape indexing using approximate nearest-neighbour search in high-dimensional spaces
- Beis, Lowe
- 1997
(Show Context)
Citation Context ...age descriptors are commonly used in object recognition, methods to mitigate these factors have been explored, such as hierarchical feature quantization [18], decision trees [19], and priority queues =-=[2]-=-. Tree-based search structures that can operate with arbitrary metrics [29, 5] remove the assumption of a vector space by exploiting the triangle inequality. However, in practice selecting useful part... |

189 | Fast Pose Estimation with ParameterSensitive Hashing
- Shakhnarovich, Viola, et al.
- 2003
(Show Context)
Citation Context ... vision researchers have recently explored approximate similarity search techniques, where a predictable loss in accuracy is sacrificed in order to allow fast queries even for high-dimensional inputs =-=[25, 24, 12, 16, 4]-=-. Methods for this problem, most notably locality-sensitive hashing (LSH) [10, 3], offer probabilistic guarantees of retrieving items within (1 + ǫ) times the optimal similarity, with query times that... |

148 | Learning the discriminative powerinvariance trade-off
- Varma, Ray
- 2005
(Show Context)
Citation Context ...el function is computable). It is thus far impossible to apply LSH and its variants to search data with a number of powerful kernels—including many kernels designed specifically for image comparisons =-=[33, 34, 30]-=-, as well as some basic well-used functions like a Gaussian RBF. Further, since visual representations are often most naturally encoded with structured inputs (e.g., sets, graphs, trees), the lack of ... |

114 | Random features for large-scale kernel machines
- Rahimi, Recht
(Show Context)
Citation Context ...ct or learn an embedding that will preserve the desired distance function, typically with the intention of mapping to a very low-dimensional space that is more easily searchable with known techniques =-=[1, 20, 28, 22, 31]-=-. These methods are related to LSH in the sense that both seek small “keys” that can be used to encode similar inputs, and often these keys exist in Hamming space. While most work with vector inputs, ... |

107 | Spectral hashing
- Weiss, Torralba, et al.
- 2008
(Show Context)
Citation Context ...ee a high probability of collision for similar examples. In a similar spirit, a number of methods show how to form low-dimensional binary embeddings that can capture more expensive distance functions =-=[1, 28, 22, 31]-=-. This line of work has shown considerable promise for a variety of image search tasks such as near-duplicate retrieval, example-based object recognition, pose estimation, and feature matching. In spi... |

99 | Semantic hashing
- Salakhutdinov, Hinton
(Show Context)
Citation Context ...ee a high probability of collision for similar examples. In a similar spirit, a number of methods show how to form low-dimensional binary embeddings that can capture more expensive distance functions =-=[1, 28, 22, 31]-=-. This line of work has shown considerable promise for a variety of image search tasks such as near-duplicate retrieval, example-based object recognition, pose estimation, and feature matching. In spi... |

88 | Bootmap: A method for efficient approximate similarity rankings
- Athitsos, Alon, et al.
- 2004
(Show Context)
Citation Context ...ee a high probability of collision for similar examples. In a similar spirit, a number of methods show how to form low-dimensional binary embeddings that can capture more expensive distance functions =-=[1, 28, 22, 31]-=-. This line of work has shown considerable promise for a variety of image search tasks such as near-duplicate retrieval, example-based object recognition, pose estimation, and feature matching. In spi... |

68 | Near duplicate image detection: min-hash and tf-idf weighting - Chum, Philbin, et al. - 2008 |

61 | Fast Image Search for Learned Metrics
- Jain, Kulis, et al.
- 2008
(Show Context)
Citation Context ... vision researchers have recently explored approximate similarity search techniques, where a predictable loss in accuracy is sacrificed in order to allow fast queries even for high-dimensional inputs =-=[25, 24, 12, 16, 4]-=-. Methods for this problem, most notably locality-sensitive hashing (LSH) [10, 3], offer probabilistic guarantees of retrieving items within (1 + ǫ) times the optimal similarity, with query times that... |

41 | Discriminant embedding for local image descriptors
- Hua, Brown, et al.
- 2007
(Show Context)
Citation Context ...ing methods on this data set since it can compute hash functions over arbitrary kernels. For feature indexing with a larger database, we provide results on the Photo Tourism data set of local patches =-=[26, 14]-=-. Finally, we experiment with the Tiny Image data set of 80 million images [27], in order to show our technique’s ability to scale to very large databases. Because our algorithm enables fast approxima... |

38 |
An Algorithm for Finding Best Matches in Logarithmic Expected Time
- Freidman, Bentley, et al.
- 1977
(Show Context)
Citation Context ...on we review related work in fast search algorithms and their application for visual search problems. Data structures using spatial partitions and recursive hyperplane decomposition (e.g., k −d trees =-=[9]-=-) provide an efficient means to search low-dimensional vector data exactly, however they are known to break down in practice for highdimensional data, and cannot provide better than a worstcase linear... |

37 |
Sub-linear indexing for large scale object recognition
- Obdržálek, Matas
- 2005
(Show Context)
Citation Context ... Since high-dimensional image descriptors are commonly used in object recognition, methods to mitigate these factors have been explored, such as hierarchical feature quantization [18], decision trees =-=[19]-=-, and priority queues [2]. Tree-based search structures that can operate with arbitrary metrics [29, 5] remove the assumption of a vector space by exploiting the triangle inequality. However, in pract... |

33 | Proximity Distribution Kernel for Geometric Context in Recognition - Ling, Soatto - 2007 |

26 | Pyramid match hashing: Sub-linear time indexing over partial correspondences
- Grauman, Darrell
- 2007
(Show Context)
Citation Context ... vision researchers have recently explored approximate similarity search techniques, where a predictable loss in accuracy is sacrificed in order to allow fast queries even for high-dimensional inputs =-=[25, 24, 12, 16, 4]-=-. Methods for this problem, most notably locality-sensitive hashing (LSH) [10, 3], offer probabilistic guarantees of retrieving items within (1 + ǫ) times the optimal similarity, with query times that... |

26 |
Satisfying general proximity / similarity queries with metric trees
- Uhlmann
- 1991
(Show Context)
Citation Context ...te these factors have been explored, such as hierarchical feature quantization [18], decision trees [19], and priority queues [2]. Tree-based search structures that can operate with arbitrary metrics =-=[29, 5]-=- remove the assumption of a vector space by exploiting the triangle inequality. However, in practice selecting useful partitioning strategies requires good heuristics, and, in spite of logarithmic que... |

13 | The law of large numbers and the central limit theorem in banach spaces. The Annals of Probability - Hoffmann-Jørgensen, Pisier - 1976 |

6 | Kernel ICA: An alternative formulation and its application to face recognition - Yang, Gao, et al. |

1 |
Mathematical Statistics and Data Aanalysis
- Rice
- 2001
(Show Context)
Citation Context ...e database itself. Our main technical contribution is to formulate the random projections necessary for LSH in kernel space. Our construction relies on an appropriate use of the central limit theorem =-=[21]-=-, which allows us to approximate a random vector using items from our database. The resulting scheme, which we call kernelized LSH (KLSH), generalizes LSH to scenarios when the feature space embedding... |