## Nearest Neighbor Search Methods for Handshape Recognition

Citations: | 4 - 1 self |

### BibTeX

@MISC{Potamias_nearestneighbor,

author = {Michalis Potamias and Vassilis Athitsos},

title = {Nearest Neighbor Search Methods for Handshape Recognition},

year = {}

}

### OpenURL

### Abstract

Gestures are an important modality for human-machine communication, and robust gesture recognition can be an important component of intelligent homes and assistive environments in general. An important aspect of gestures is handshape. Handshapes can hold important information about the meaning of a gesture, for example in sign languages, or about the intent of an action, for example in manipulative gestures or in virtual reality interfaces. At the same time, recognizing handshape can be a very challenging task, because the same handshape can look very different in different images, depending on the 3D orientation of the hand and the viewpoint of the camera. In this paper we examine a database approach for handshape classification, whereby a large database of tens of thousands of images is used to represent the wide variability of handshape appearance. Efficient and accurate indexing methods are important in such a database approach, to ensure that the system can match every incoming image to the large number of database images at interactive times. In this paper we examine the use of embedding-based and hash table-based indexing methods for handshape recognition, and we experimentally compare these two approaches on the task of recognizing 20 handshapes commonly used in American Sign Language (ASL).

### Citations

3464 | A computational approach to edge detection
- Canny
- 1986
(Show Context)
Citation Context ... chamfer distance operates on edge images. The synthetic images generated by Poser can be rendered directly as edge images by the software. For the test images we simply apply the Canny edge detector =-=[11]-=-. On an AMD Athlon processor running at 2.0GHz, we can compute on average 715 chamfer distances per second. Consequently, finding the nearest neighbors of each test image using brute force search, whi... |

742 | Improved boosting algorithms using confidence-rated predictions
- Schapire, Singer
- 1999
(Show Context)
Citation Context ...lable from the machine learning community. In particular, embedding optimization can be casted as the machine learning problem of optimizing a binary classifier, and boosting methods such as AdaBoost =-=[27]-=- can be employed for embedding optimization [3], as described in the next paragraphs. Suppose we have an embedding F with the following property: for any Q,A, B ∈ X (where X is our space of edge image... |

467 | Similarity Search in High Dimensions via Hashing
- GIONIS, INDYK, et al.
- 1999
(Show Context)
Citation Context ...nt) from HDBH we can define k-bit hash functions gi as follows: gi(X) = (hi1(X), hi2(X), . . . , hik(X)) . (12) This way, indexing and retrieval can be performed as in Locality Sensitive Hashing (LSH)=-=[16]-=-, by: • Choosing parameters k and l. • Constructing l k-bit hash tables, and storing each database object to the appropriate l buckets. • Comparing the input image with the database images found in th... |

459 | The geometry of graphs and some of its algorithmic applications
- Linial, London, et al.
- 1995
(Show Context)
Citation Context ... F(X) = (D(X, R1), D(X, R2), ..., D(X, Rd)) . (3) where D is the chamfer distance, defined in Equation 1, and X is an edge image. The function F turns out to be a special case of Lipschitz embeddings =-=[9, 23]-=-. We define the approximate chamfer distance D ′ between two edge images X1 and X2 to be the L1 distance between F(X1) and F(X2): D ′ (A,B) = dX |D(X1, Ri) − D(X2, Ri)| . (4) i=1 The actual value of D... |

435 | FastMap: A Fast Algorithm for Indexing, Data-Mining and Visualization of Traditional and Multimedia Datasets
- Faloutsos, Lin
- 1995
(Show Context)
Citation Context ...e obtained by embedding objects into another space with a more efficient distance measure. Several methods have been proposed for embedding arbitrary spaces into a Euclidean or pseudo-Euclidean space =-=[3, 4, 9, 14, 21, 35]-=-. These methods are indeed applicable to our setting. In this paper we focus on the BoostMap embedding method [3] and we show that this method can be successfully employed for efficient matching of ha... |

294 | Data structures and algorithms for nearest neighbor search in general metric spaces
- Yianilos
- 1993
(Show Context)
Citation Context ...es) is not an Lp measure. A number of nearest neighbor methods can be applied for indexing arbitrary metric spaces; the reader is referred to [20] for surveys of such methods. As an example, VP-trees =-=[39]-=- and metric trees [33] hierarchically partition the database into a tree structure by splitting, at each node, the set of objects based on their distances to pivot objects. However, while such methods... |

280 |
On Lipschitz embedding of finite metric spaces in Hilbert space
- Bourgain
- 1985
(Show Context)
Citation Context ...e obtained by embedding objects into another space with a more efficient distance measure. Several methods have been proposed for embedding arbitrary spaces into a Euclidean or pseudo-Euclidean space =-=[3, 4, 9, 14, 21, 35]-=-. These methods are indeed applicable to our setting. In this paper we focus on the BoostMap embedding method [3] and we show that this method can be successfully employed for efficient matching of ha... |

269 |
Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases
- Böhm, Berchtold, et al.
(Show Context)
Citation Context ... task of finding, given the input image, the best matching database images. Various methods have been employed for speeding up nearest neighbor retrieval. Comprehensive reviews on the subject include =-=[8, 19, 20]-=-. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., [22, 32, 36]. However, that family of approaches is not applicable ... |

267 | Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions
- Andoni, Indyk
- 2008
(Show Context)
Citation Context ... framework to a specific space and distance measure requires identifying a locality sensitive family [16]. Such families have been identified for certain spaces, such as vector spaces with Lp metrics =-=[2, 16]-=-, or strings with a substitution-based distance measure [1, 10]. In Euclidean space R d , the time complexity of retrieval using LSH is linear in the dimensionality d and sublinear in the number n of ... |

203 | Fast pose estimation with parameter-sensitive hashing
- Shakhnarovich, Viola, et al.
- 2003
(Show Context)
Citation Context ...ng (LSH) is an approximate nearest neighbor method that is based on hash tables. LSH has been shown theoretically to scale well with the number of dimensions and has produced good results in practice =-=[17, 28]-=-. However, LSH cannot be applied to arbitrary distance measures, and there is no existing method that allows applying LSH to the chamfer distance. An alternative hash-based method that can be applied ... |

195 |
H.C.: Parametric correspondence and chamfer matching: two new techniques for image matching
- Barrow, Tenenbaum, et al.
- 1977
(Show Context)
Citation Context ...measures that are most meaningful in comparing hand images are typically non-Euclidean, nonmetric, and computationally expensive. Examples of such nonmetric distance measures are the chamfer distance =-=[6]-=-, shape context matching [7, 31], and distance measures based on the Viterbi algorithm [31]. • The majority of database indexing methods are designed for Euclidean distance measures or metric distance... |

149 | Index-driven similarity search in metric spaces
- Hjaltason, Samet
- 2003
(Show Context)
Citation Context ... task of finding, given the input image, the best matching database images. Various methods have been employed for speeding up nearest neighbor retrieval. Comprehensive reviews on the subject include =-=[8, 19, 20]-=-. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., [22, 32, 36]. However, that family of approaches is not applicable ... |

120 | Towards 3d hand tracking using a deformable model
- Heap, Hogg
- 1996
(Show Context)
Citation Context ...elatively low classification accuracy, can be immensely useful in identifying a relatively small set of likely hypotheses. Such a set of hypotheses can subsequently be refined: • using a hand tracker =-=[25, 18, 24, 29, 30, 38]-=-, • using domain-specific knowledge, such as ASL linguistic constraints, or • using knowledge of a specific protocol for human-computer communication, that can place constraints on the current handsha... |

100 | Shape context and chamfer matching in cluttered scenes
- Thayananthan, Stenger, et al.
- 2003
(Show Context)
Citation Context ...ngful in comparing hand images are typically non-Euclidean, nonmetric, and computationally expensive. Examples of such nonmetric distance measures are the chamfer distance [6], shape context matching =-=[7, 31]-=-, and distance measures based on the Viterbi algorithm [31]. • The majority of database indexing methods are designed for Euclidean distance measures or metric distance measures (i.e., distance measur... |

93 | Capturing natural hand articulation
- Wu, Lin, et al.
- 2001
(Show Context)
Citation Context ...elatively low classification accuracy, can be immensely useful in identifying a relatively small set of likely hypotheses. Such a set of hypotheses can subsequently be refined: • using a hand tracker =-=[25, 18, 24, 29, 30, 38]-=-, • using domain-specific knowledge, such as ASL linguistic constraints, or • using knowledge of a specific protocol for human-computer communication, that can place constraints on the current handsha... |

85 | Properties of embedding methods for similarity searching in metric spaces
- Hjaltason, Samet
(Show Context)
Citation Context ... task of finding, given the input image, the best matching database images. Various methods have been employed for speeding up nearest neighbor retrieval. Comprehensive reviews on the subject include =-=[8, 19, 20]-=-. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., [22, 32, 36]. However, that family of approaches is not applicable ... |

82 | J.(2001):Effective large-scale sequence comparison by locality-sensitive hashing. Bioinformatics
- Buhler
(Show Context)
Citation Context ...entifying a locality sensitive family [16]. Such families have been identified for certain spaces, such as vector spaces with Lp metrics [2, 16], or strings with a substitution-based distance measure =-=[1, 10]-=-. In Euclidean space R d , the time complexity of retrieval using LSH is linear in the dimensionality d and sublinear in the number n of database objects [2]. At the same time, LSH cannot be applied t... |

78 | Fast Contour Matching Using Approximate Earth Mover‟s Distance
- Grauman, Darrell
- 2004
(Show Context)
Citation Context ...ng (LSH) is an approximate nearest neighbor method that is based on hash tables. LSH has been shown theoretically to scale well with the number of dimensions and has produced good results in practice =-=[17, 28]-=-. However, LSH cannot be applied to arbitrary distance measures, and there is no existing method that allows applying LSH to the chamfer distance. An alternative hash-based method that can be applied ... |

75 | 3D hand pose reconstruction using specialized mappings
- ROSALES, ATHITSOS, et al.
- 2001
(Show Context)
Citation Context ...ed in this paper can be used (among other things) to automate the initialization and error recovery of a hand tracker. A regression system that estimates hand pose from a single image is described in =-=[26]-=-. However, that method assumes that the hand silhouette is correctly identified in the input image, whereas such precise hand detection is often unrealistic to assume in a realworld application. Anoth... |

75 | View-independent recognition of hand postures
- Wu, Huang
- 2000
(Show Context)
Citation Context ... an input to our system, but we do not require precise separation of the hand silhouette from the background. Another family of methods for hand shape classification are appearancebased methods, like =-=[15, 37]-=-. Such methods are typically limited to estimating 2D hand pose from a limited number of viewpoints. In contrast, the method described in this paper can handle arbitrary viewpoints. Our system uses th... |

73 | Model-based hand tracking using a hierarchical Bayesian filter
- Stenger, Thayananthan, et al.
- 2006
(Show Context)
Citation Context ...elatively low classification accuracy, can be immensely useful in identifying a relatively small set of likely hypotheses. Such a set of hypotheses can subsequently be refined: • using a hand tracker =-=[25, 18, 24, 29, 30, 38]-=-, • using domain-specific knowledge, such as ASL linguistic constraints, or • using knowledge of a specific protocol for human-computer communication, that can place constraints on the current handsha... |

66 | Computer vision for computer games
- Freeman, Tanakan, et al.
- 1996
(Show Context)
Citation Context ... an input to our system, but we do not require precise separation of the hand silhouette from the background. Another family of methods for hand shape classification are appearancebased methods, like =-=[15, 37]-=-. Such methods are typically limited to estimating 2D hand pose from a limited number of viewpoints. In contrast, the method described in this paper can handle arbitrary viewpoints. Our system uses th... |

53 | Cluster-preserving embedding of proteins
- Hristescu, Farach-Colton
- 1999
(Show Context)
Citation Context ...e obtained by embedding objects into another space with a more efficient distance measure. Several methods have been proposed for embedding arbitrary spaces into a Euclidean or pseudo-Euclidean space =-=[3, 4, 9, 14, 21, 35]-=-. These methods are indeed applicable to our setting. In this paper we focus on the BoostMap embedding method [3] and we show that this method can be successfully employed for efficient matching of ha... |

52 | Trading quality for time with nearest neighbor search
- Weber, Boehm
(Show Context)
Citation Context ...retrieval. Comprehensive reviews on the subject include [8, 19, 20]. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., =-=[22, 32, 36]-=-. However, that family of approaches is not applicable in our setting, since the chamfer distance (i.e., the distance measure that we use for comparing hand images) is not an Lp measure. A number of n... |

48 | Clustering for approximate similarity search in high-dimensional spaces
- Li, Chang, et al.
(Show Context)
Citation Context ...retrieval. Comprehensive reviews on the subject include [8, 19, 20]. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., =-=[22, 32, 36]-=-. However, that family of approaches is not applicable in our setting, since the chamfer distance (i.e., the distance measure that we use for comparing hand images) is not an Lp measure. A number of n... |

44 |
Real-time 3d hand posture estimation based on 2-d appearance retrieval using monocular camera. ratfg-rts
- Shimada, Kimura, et al.
- 2001
(Show Context)
Citation Context |

40 | Using multiple cues for hand tracking and model refinement
- Lu, Metaxas, et al.
- 2003
(Show Context)
Citation Context |

36 | Visual Analysis of High DOF Articulated Object with Application to Hand Tracking
- Rehg, Kanade
- 1993
(Show Context)
Citation Context |

26 | An index structure for data mining and clustering
- Wang, Wang, et al.
(Show Context)
Citation Context |

24 | K.: Vq-index: an index structure for similarity searching in multimedia databases
- Tuncel, Ferhatosmanoglu, et al.
- 2002
(Show Context)
Citation Context ...retrieval. Comprehensive reviews on the subject include [8, 19, 20]. A large amount of work focuses on efficient nearest neighbor retrieval in multidimensional vector spaces using an Lp metric, e.g., =-=[22, 32, 36]-=-. However, that family of approaches is not applicable in our setting, since the chamfer distance (i.e., the distance measure that we use for comparing hand images) is not an Lp measure. A number of n... |

21 | Query-sensitive embeddings - Athitsos, Hadjieleftheriou, et al. - 2005 |

18 | Automatic 2d hand tracking in video sequences in
- Yuan, Sclaroff, et al.
- 2005
(Show Context)
Citation Context ...rom video sequences of a native ASL signer either performing individual handshapes in isolation or signing in ASL. The hand locations were extracted from those sequences using the method described in =-=[40]-=-. The test images are obtained from the original frames by extracting the subwindow corresponding to the hand region, and then performing the same normalization that we perform for database images, so... |

16 | D.: Regression-based hand pose estimation from multiple cameras
- Campos, Murray
(Show Context)
Citation Context ...he hand silhouette is correctly identified in the input image, whereas such precise hand detection is often unrealistic to assume in a realworld application. Another regression method is presented at =-=[13]-=-, but that method requires that the hand be simultaneously visible from multiple cameras. The database approach described here has the advantage that it only requires a single camera, and it can toler... |

14 | Boostmap: An embedding method for efficient nearest neighbor retrieval
- Athitsos, Alon, et al.
- 2008
(Show Context)
Citation Context ... images. In this paper we examine the use of recently proposed embeddingbased and hash table-based indexing methods for handshape recognition. In particular, we consider the BoostMap embedding method =-=[3]-=- and Distance-Based Hashing (DBH) [5]. We discuss how to apply those methods for efficient retrieval of hand images, and we compare the performance of both methods on the task recognizing 20 handshape... |

14 | Detecting objects of variable shape structure with hidden state shape models
- Wang, Athitsos, et al.
(Show Context)
Citation Context ...rce search. While the experiments show the need for more research, to design image matching methods that are more accurate that the chamfer distance (some recent progress on that topic is reported at =-=[34]-=-), the experiments also illustrate the power of BoostMap and DBH as indexing methods. BoostMap yields a classification time that is about three orders of magnitude faster than that of brute-force sear... |

13 | Efficient algorithms for substring near neighbor problem
- Andoni, Indyk
- 2006
(Show Context)
Citation Context ...entifying a locality sensitive family [16]. Such families have been identified for certain spaces, such as vector spaces with Lp metrics [2, 16], or strings with a substitution-based distance measure =-=[1, 10]-=-. In Euclidean space R d , the time complexity of retrieval using LSH is linear in the dimensionality d and sublinear in the number n of database objects [2]. At the same time, LSH cannot be applied t... |

13 | Nearest neighbor retrieval using distance-based hashing
- Athitsos, Potamias, et al.
(Show Context)
Citation Context ... use of recently proposed embeddingbased and hash table-based indexing methods for handshape recognition. In particular, we consider the BoostMap embedding method [3] and Distance-Based Hashing (DBH) =-=[5]-=-. We discuss how to apply those methods for efficient retrieval of hand images, and we compare the performance of both methods on the task recognizing 20 handshapes commonly used in American Sign Lang... |

6 |
Satisfying general proximity/similarity queries with metric trees
- Uhlman
- 1991
(Show Context)
Citation Context ...re. A number of nearest neighbor methods can be applied for indexing arbitrary metric spaces; the reader is referred to [20] for surveys of such methods. As an example, VP-trees [39] and metric trees =-=[33]-=- hierarchically partition the database into a tree structure by splitting, at each node, the set of objects based on their distances to pivot objects. However, while such methods can offer theoretical... |