## Visual hand tracking using nonparametric belief propagation (2004)

### Cached

### Download Links

Venue: | Propagation,” IEEE Workshop on Generative Model Based Vision |

Citations: | 53 - 1 self |

### BibTeX

@INPROCEEDINGS{Sudderth04visualhand,

author = {Erik B. Sudderth and Michael I. M and William T. Freeman and Alan S. Willsky},

title = {Visual hand tracking using nonparametric belief propagation},

booktitle = {Propagation,” IEEE Workshop on Generative Model Based Vision},

year = {2004},

pages = {189}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract — This paper develops probabilistic methods for visual tracking of a three-dimensional geometric hand model from monocular image sequences. We consider a redundant representation in which each model component is described by its position and orientation in the world coordinate frame. A prior model is then defined which enforces the kinematic constraints implied by the model’s joints. We show that this prior has a local structure, and is in fact a pairwise Markov random field. Furthermore, our redundant representation allows color and edge-based likelihood measures, such as the Chamfer distance, to be similarly decomposed in cases where there is no self–occlusion. Given this graphical model of hand kinematics, we may track the hand’s motion using the recently proposed nonparametric belief propagation (NBP) algorithm. Like particle filters, NBP approximates the posterior distribution over hand configurations as a collection of samples. However, NBP uses the graphical structure to greatly reduce the dimensionality of these distributions, providing improved robustness. Several methods are used to improve NBP’s computational efficiency, including a novel KD-tree based method for fast Chamfer distance evaluation. We provide simulations showing that NBP may be used to refine inaccurate model initializations, as well as track hand motion through extended image sequences. I.

### Citations

2339 |
Density estimation for statistics and data analysis
- Silverman
- 1986
(Show Context)
Citation Context ... 3) Compute an importance weight for each sample x (ℓ) i : w (ℓ) i ∝ pE(y|x(ℓ) ), normalize the orientation r(ℓ) i . ) � i )pC(y|x(ℓ) i j∈ΓS (i) mji(x (ℓ) i ) 4) Use a bandwidth selection method (see =-=[18]-=-) to construct a kernel density estimate ˆp(xi | y) from {x (ℓ) i , w (ℓ) i }M ℓ=1. Alg. 1. NBP update of the estimated marginal distribution ˆp(xi | y). incoming messages from neighboring nodes with ... |

1174 |
Multidimensional binary search trees used for associative searching
- Bentley
- 1975
(Show Context)
Citation Context ...lementation of NBP on this model, as well as several methods for improving computational efficiency. These include a novel method for fast orientation–based Chamfer distance evaluation using KD–trees =-=[2]-=-. We conclude with simulations demonstrating that NBP can refine noisy initializations in single frames, as well as track hand motion over two extended sequences. A. Structural Model II. GEOMETRIC HAN... |

399 | Statistical color models with application to skin detection
- Jones, Rehg
- 2002
(Show Context)
Citation Context ...iscussion of the generative model underlying this likelihood function, see [23]. 4 B. Silhouette Matching Using Skin Color Statistics Skin colored pixels are well known to have predictable statistics =-=[9]-=-, and thus provide a powerful cue for hand tracking. We model the color distribution pskin of skin pixels by a single Gaussian in RGB space, with mean and covariance estimated from hand–selected train... |

222 | Nonparametric belief propagation
- Sudderth, Ihler, et al.
(Show Context)
Citation Context ...s are inapplicable. Instead, these trackers are based on recently proposed extensions of particle filters to general graphs: mean field Monte Carlo in [24], and nonparametric belief propagation (NBP) =-=[8, 21]-=- in [17]. In this paper, we show that NBP may be used to track a three–dimensional geometric model of the hand. To derivesFig. 1. Projected edges (top row) and silhouettes (bottom row) for two configu... |

179 | Partitioned sampling, articulated objects, and interface-quality hand tracking
- MacCormick, Isard
- 2000
(Show Context)
Citation Context ...ost significant modes, and the tracker can suffer catastrophic failures. This problem 1 has motivated previous authors to consider simplified models which only allow a limited range of object motions =-=[11]-=-, as well as sophisticated prior models which better predict the object’s dynamics [16, 26]. Deterministic nonparametric approximations are also possible, as demonstrated by a recently proposed tree–b... |

176 | Implicit probabilistic models of human motion for synthesis and tracking
- SIDENBLADH, BLACK, et al.
(Show Context)
Citation Context ... has motivated previous authors to consider simplified models which only allow a limited range of object motions [11], as well as sophisticated prior models which better predict the object’s dynamics =-=[16, 26]-=-. Deterministic nonparametric approximations are also possible, as demonstrated by a recently proposed tree–based estimator [20] which defines a multiscale discretization of the state space. This appr... |

137 | Estimating 3D hand pose from a cluttered image
- Athitsos, Sclaroff
- 2003
(Show Context)
Citation Context ...ng rules are very likely to miss important hand configurations. Given the difficulties in approximating high–dimensional distributions, some authors have proposed replacing tracking by classification =-=[1, 15]-=-, where classes correspond to some discretization of allowable hand configurations. These methods are most appropriate in applications such as sign language recognition, where only a small set of pose... |

123 | Finding and tracking people from the bottom up
- Ramanan, Forsyth
- 2003
(Show Context)
Citation Context ...ructure within the posterior distribution. This structure can be described using a graphical model, or Bayesian network. Graphical models have been used to track view–based human body representations =-=[13]-=-, contour models of restricted hand configurations [4], view–based 2.5D “cardboard” models of hands and people [24], and a full 3D kinematic human body model [17]. Because the variables in these graph... |

106 | Fast Pose Estimation with Parameter Sensitive Hashing
- Shakhnarovich, Viola, et al.
- 2003
(Show Context)
Citation Context ...ng rules are very likely to miss important hand configurations. Given the difficulties in approximating high–dimensional distributions, some authors have proposed replacing tracking by classification =-=[1, 15]-=-, where classes correspond to some discretization of allowable hand configurations. These methods are most appropriate in applications such as sign language recognition, where only a small set of pose... |

101 | Finding deformable shapes using loopy belief propagation
- Coughlan, Ferreira
- 2002
(Show Context)
Citation Context ...re can be described using a graphical model, or Bayesian network. Graphical models have been used to track view–based human body representations [13], contour models of restricted hand configurations =-=[4]-=-, view–based 2.5D “cardboard” models of hands and people [24], and a full 3D kinematic human body model [17]. Because the variables in these graphical models are continuous, and discretization is intr... |

98 | Human body model acquisition and tracking using voxel data
- Mikic, Trivedi, et al.
- 2003
(Show Context)
Citation Context ...f freedom, which are only indirectly related to the observed images, exact representation of the posterior distribution over model configurations is intractable. Extended and unscented Kalman filters =-=[12, 14, 19]-=- approximate the posterior by a single Gaussian, and update these approximations via a linearization of the measurement process. However, because many different hand configurations may approximately m... |

96 | Pampas: Real-valued graphical models for computer vision
- Isard
- 2003
(Show Context)
Citation Context ...s are inapplicable. Instead, these trackers are based on recently proposed extensions of particle filters to general graphs: mean field Monte Carlo in [24], and nonparametric belief propagation (NBP) =-=[8, 21]-=- in [17]. In this paper, we show that NBP may be used to track a three–dimensional geometric model of the hand. To derivesFig. 1. Projected edges (top row) and silhouettes (bottom row) for two configu... |

77 | A general algorithm for approximate inference and its application to hybrid Bayes nets
- Koller, Lerner, et al.
- 1999
(Show Context)
Citation Context ...In the future, we hope to replace this manual initialization by automatic image– based feature detectors. D. Message Propagation and Scheduling To derive the message propagation rule, as suggested by =-=[10]-=- we rewrite the message update equation (15) in terms of the Given M weighted samples {x (ℓ) i , w (ℓ) i }Mℓ=1 from ˆp(xi | y), and the incoming message mji (xi) used to construct ˆp(xi | y): as ¯w (ℓ... |

68 |
Filtering Using a Tree-Based Estimator
- Stenger, Thayananthan, et al.
- 2003
(Show Context)
Citation Context ...phisticated prior models which better predict the object’s dynamics [16, 26]. Deterministic nonparametric approximations are also possible, as demonstrated by a recently proposed tree–based estimator =-=[20]-=- which defines a multiscale discretization of the state space. This approach achieves computational savings by approximating image likelihoods as piecewise constant at coarse scales of the discretizat... |

66 | Digiteyes: Vision-based hand tracking for human-computer interaction
- Rehg, Kanade
- 1994
(Show Context)
Citation Context ...f freedom, which are only indirectly related to the observed images, exact representation of the posterior distribution over model configurations is intractable. Extended and unscented Kalman filters =-=[12, 14, 19]-=- approximate the posterior by a single Gaussian, and update these approximations via a linearization of the measurement process. However, because many different hand configurations may approximately m... |

54 | Attractive people: Assembling loose-limbed models using non-parametric belief propagation
- Sigal, Isard, et al.
- 2003
(Show Context)
Citation Context ...view–based human body representations [13], contour models of restricted hand configurations [4], view–based 2.5D “cardboard” models of hands and people [24], and a full 3D kinematic human body model =-=[17]-=-. Because the variables in these graphical models are continuous, and discretization is intractable for three–dimensional models, most traditional graphical inference algorithms are inapplicable. Inst... |

52 | Model-based 3D tracking of an articulated hand
- Stenger, Mendonça, et al.
- 2001
(Show Context)
Citation Context ...f freedom, which are only indirectly related to the observed images, exact representation of the posterior distribution over model configurations is intractable. Extended and unscented Kalman filters =-=[12, 14, 19]-=- approximate the posterior by a single Gaussian, and update these approximations via a linearization of the measurement process. However, because many different hand configurations may approximately m... |

40 | Efficient multiscale sampling from products of Gaussian mixtures
- Ihler, Sudderth, et al.
- 2003
(Show Context)
Citation Context ...ors ΓS(i), and temporal neighbors ΓT (i): 1) Draw M independent samples {x (ℓ) i }M ℓ=1 from the product x (ℓ) i ∼ � j∈Γ T (i) m ji (xi) � k∈Γ K (i) m ki (xi) using the multiscale sampling methods of =-=[7]-=-. 2) For each x (ℓ) i = (q(ℓ) i , r (ℓ) i 3) Compute an importance weight for each sample x (ℓ) i : w (ℓ) i ∝ pE(y|x(ℓ) ), normalize the orientation r(ℓ) i . ) � i )pC(y|x(ℓ) i j∈ΓS (i) mji(x (ℓ) i ) ... |

21 | Automatic camera calibration from a single manhattan image
- Deutscher, Isard, et al.
- 2002
(Show Context)
Citation Context ...ures of the hand tracking application. The hand tracking application is complicated by the fact that the orientation component ri of xi = (qi, ri) is an element of the rotation group SO(3). Following =-=[5, 17]-=-, we represent orientations as unit quaternions, and use a linearized approximation when constructing density estimates. Any sampled orientations may be projected back to SO(3) by normalizing the corr... |

5 |
The algebraic properties of second-order surfaces
- Blinn
- 1997
(Show Context)
Citation Context ...spective projection of a quadric surface is a conic, one can efficiently determine the image points lying on the boundary or silhouette of the projection of any three– dimensional model configuration =-=[3, 19]-=-. Figure 1 shows the edges and silhouettes corresponding to two different configurations of the hand model, each of which is seen from three different viewpoints. Because our model is designed for est... |