## Distributed occlusion reasoning for tracking with nonparametric belief propagation (2004)

### Cached

### Download Links

- [books.nips.cc]
- [ssg.mit.edu]
- [ssg.mit.edu]
- [www.ai.mit.edu]
- [people.csail.mit.edu]
- [cs.brown.edu]
- [cs.brown.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In NIPS |

Citations: | 54 - 0 self |

### BibTeX

@INPROCEEDINGS{Sudderth04distributedocclusion,

author = {Erik B. Sudderth and Michael I. M and William T. Freeman and Alan S. Willsky},

title = {Distributed occlusion reasoning for tracking with nonparametric belief propagation},

booktitle = {In NIPS},

year = {2004},

pages = {1369--1376},

publisher = {MIT Press}

}

### Years of Citing Articles

### OpenURL

### Abstract

We describe a three–dimensional geometric hand model suitable for visual tracking applications. The kinematic constraints implied by the model’s joints have a probabilistic structure which is well described by a graphical model. Inference in this model is complicated by the hand’s many degrees of freedom, as well as multimodal likelihoods caused by ambiguous image measurements. We use nonparametric belief propagation (NBP) to develop a tracking algorithm which exploits the graph’s structure to control complexity, while avoiding costly discretization. While kinematic constraints naturally have a local structure, self– occlusions created by the imaging process lead to complex interpendencies in color and edge–based likelihood functions. However, we show that local structure may be recovered by introducing binary hidden variables describing the occlusion state of each pixel. We augment the NBP algorithm to infer these occlusion variables in a distributed fashion, and then analytically marginalize over them to produce hand position estimates which properly account for occlusion events. We provide simulations showing that NBP may be used to refine inaccurate model initializations, as well as track hand motion through extended image sequences. 1

### Citations

2209 |
Density Estimation for Statistics and Data Analysis
- Silverman
- 1986
(Show Context)
Citation Context ...ional component, each sample is assigned a weight equal to the product of the color and edge likelihoods with any structural messages. Finally, the computationally efficient “rule of thumb” heuristic =-=[17]-=- is used to set the bandwidth of Gaussian kernels placed around each sample. To derive BP updates for the occlusion masks zi, we first cluster (xi,zi) for each hand component so that p(x t ,z t | y t ... |

415 | Constructing free energy approximations and Generalised Belief Propagation algorithms. TR-2002-35 Mitubishi Electronic Reseach Labs (www.merl.com
- Yedidia, Freeman, et al.
- 2002
(Show Context)
Citation Context ... Markov random field: p(x | y) ∝ � ψi,j (xi,xj) � ψi (xi,y) (12) (i,j)∈E Hand tracking can thus be posed as inference in a graphical model, a problem we propose to solve using belief propagation (BP) =-=[15]-=-. At each BP iteration, some node i ∈ V calculates a message mij (xj) to be sent to a neighbor j ∈ Γ(i) � {j | (i,j) ∈ E}: m n � ij (xj) ∝ ψj,i (xj,xi) ψi (xi,y) � m n−1 ki (xi) dxi (13) xi i=1 i∈V k∈... |

372 | Statistical Color Models with Application to Skin Detection - Jones, Rehg - 1998 |

215 | Nonparametric Belief Propagation
- Sudderth, Ihler, et al.
- 2003
(Show Context)
Citation Context ...ms are inapplicable. Instead, these trackers are based on recently proposed extensions of particle filters to general graphs: mean field Monte Carlo in [9], and nonparametric belief propagation (NBP) =-=[11, 12]-=- in [10]. In this paper, we show that NBP may be used to track a three–dimensional geometric model of the hand. To derive a graphical model for the tracking problem, we consider a redundant local repr... |

174 | Partitioned sampling, articulated objects, and interface quality hand tracking
- MacCormick, Isard
(Show Context)
Citation Context ...2, 3] have difficulties with the multimodal uncertainties produced by ambiguous image evidence. This has motived many researchers to consider nonparametric representations, including particle filters =-=[4, 5]-=- and deterministic multiscale discretizations [6]. However, the hand’s high dimensionality can cause these trackers to suffer catastrophic failures, requiring the use of models which limit the hand’s ... |

120 | Finding and tracking people from the bottom up
- Ramanan, Forsyth
- 2003
(Show Context)
Citation Context ...(right) about the vertical axis. to describe the posterior distribution’s statistical structure using a graphical model. Graphical models have been used to track view–based human body representations =-=[7]-=-, contour models of restricted hand configurations [8], view–based 2.5D “cardboard” models of hands and people [9], and a full 3D kinematic human body model [10]. Because the variables in these graphi... |

100 | Finding Deformable Shapes Using Loopy Belief Propagation
- Coughlan, Ferreira
- 2002
(Show Context)
Citation Context ...rior distribution’s statistical structure using a graphical model. Graphical models have been used to track view–based human body representations [7], contour models of restricted hand configurations =-=[8]-=-, view–based 2.5D “cardboard” models of hands and people [9], and a full 3D kinematic human body model [10]. Because the variables in these graphical models are continuous, and discretization is intra... |

94 | PAMPAS: Real-Valued Graphical Models for Computer Vision
- Isard
- 2003
(Show Context)
Citation Context ...ms are inapplicable. Instead, these trackers are based on recently proposed extensions of particle filters to general graphs: mean field Monte Carlo in [9], and nonparametric belief propagation (NBP) =-=[11, 12]-=- in [10]. In this paper, we show that NBP may be used to track a three–dimensional geometric model of the hand. To derive a graphical model for the tracking problem, we consider a redundant local repr... |

90 | Capturing natural hand articulation
- Wu, Lin, et al.
(Show Context)
Citation Context ...2, 3] have difficulties with the multimodal uncertainties produced by ambiguous image evidence. This has motived many researchers to consider nonparametric representations, including particle filters =-=[4, 5]-=- and deterministic multiscale discretizations [6]. However, the hand’s high dimensionality can cause these trackers to suffer catastrophic failures, requiring the use of models which limit the hand’s ... |

74 | A general algorithm for approximate inference and its application to hybrid bayes nets
- Koller, Lerner, et al.
- 1998
(Show Context)
Citation Context ...figurations of xi as before, and reweighting them using these occlusion–sensitive likelihood functions. 4.3 Message Propagation To derive the propagation rule for non–occlusion edges, as suggested by =-=[18]-=- we rewrite the message update equation (13) in terms of the marginal distribution ˆp(xi | y): m n � ij (xj) = α ψj,i (xj,xi) xi ˆpn−1 (xi | y) m n−1 ji (xi) dxi (17) Our explicit use of the current m... |

66 |
Filtering using a tree-based estimator
- Stenger, Thayananthan, et al.
- 2005
(Show Context)
Citation Context ...nties produced by ambiguous image evidence. This has motived many researchers to consider nonparametric representations, including particle filters [4, 5] and deterministic multiscale discretizations =-=[6]-=-. However, the hand’s high dimensionality can cause these trackers to suffer catastrophic failures, requiring the use of models which limit the hand’s motion [4] or sophisticated prior models of hand ... |

61 | DigitEyes: vision-based hand tracking for humancomputer interaction
- Rehg, Kanade
- 1994
(Show Context)
Citation Context ... models have many (roughly 26) degrees of freedom, exact representation of the posterior distribution over model configurations is intractable. Trackers based on extended and unscented Kalman filters =-=[2, 3]-=- have difficulties with the multimodal uncertainties produced by ambiguous image evidence. This has motived many researchers to consider nonparametric representations, including particle filters [4, 5... |

54 | Tracking articulated body by dynamic markov network
- Wu, Hua, et al.
(Show Context)
Citation Context ...model. Graphical models have been used to track view–based human body representations [7], contour models of restricted hand configurations [8], view–based 2.5D “cardboard” models of hands and people =-=[9]-=-, and a full 3D kinematic human body model [10]. Because the variables in these graphical models are continuous, and discretization is intractable for three–dimensional models, most traditional graphi... |

50 | R.: Model-based 3d tracking of an articulated hand
- Stenger, Mendona, et al.
(Show Context)
Citation Context ... models have many (roughly 26) degrees of freedom, exact representation of the posterior distribution over model configurations is intractable. Trackers based on extended and unscented Kalman filters =-=[2, 3]-=- have difficulties with the multimodal uncertainties produced by ambiguous image evidence. This has motived many researchers to consider nonparametric representations, including particle filters [4, 5... |

50 | Visual hand tracking using nonparametric belief propagation. CVPR
- Sudderth, Mandel, et al.
- 2004
(Show Context)
Citation Context ...representation. We also provide a local decomposition of the likelihood function which properly handles occlusion in a distributed fashion, a significant improvement over our earlier tracking results =-=[13]-=-. We conclude with simulations demonstrating our algorithm’s robustness to occlusions. 2 Geometric Hand Modeling Structurally, the hand is composed of sixteen approximately rigid components: three pha... |

39 | Efficient multiscale sampling from products of Gaussian mixtures
- Ihler, Sudderth, et al.
- 2003
(Show Context)
Citation Context ...ere is no self–occlusion. First, M samples are drawn from the product of the incoming kinematic and temporal messages, which are Gaussian mixtures. We use a recently proposed multiscale Gibbs sampler =-=[16]-=- to efficiently draw accurate (albeit approximate) samples, while avoiding the exponential cost associated with direct sampling (a product of d M–Gaussian mixtures contains M d Gaussians). Following n... |

4 |
modeling, analysis, and recognition
- Hand
- 2001
(Show Context)
Citation Context ...uction Accurate visual detection and tracking of three–dimensional articulated objects is a challenging problem with applications in human–computer interfaces, motion capture, and scene understanding =-=[1]-=-. In this paper, we develop a probabilistic method for tracking a geometric hand model from monocular image sequences. Because articulated hand models have many (roughly 26) degrees of freedom, exact ... |

1 |
Attractive people: Assembling loose– limbed models using nonparametric belief propagation
- Sigal, Isard, et al.
- 2003
(Show Context)
Citation Context ...k view–based human body representations [7], contour models of restricted hand configurations [8], view–based 2.5D “cardboard” models of hands and people [9], and a full 3D kinematic human body model =-=[10]-=-. Because the variables in these graphical models are continuous, and discretization is intractable for three–dimensional models, most traditional graphical inference algorithms are inapplicable. Inst... |