## Tracking People with Twists and Exponential Maps (1998)

### Cached

### Download Links

- [www.cs.berkeley.edu]
- [www.cs.berkeley.edu]
- [www.eecs.berkeley.edu]
- [www.eecs.berkeley.edu]
- [www.ece.northwestern.edu]
- [www.cs.berkeley.edu]
- DBLP

### Other Repositories/Bibliography

Citations: | 382 - 3 self |

### BibTeX

@MISC{Bregler98trackingpeople,

author = {Christoph Bregler and Jitendra Malik},

title = {Tracking People with Twists and Exponential Maps},

year = {1998}

}

### Years of Citing Articles

### OpenURL

### Abstract

This paper demonstrates a new visual motion estimation technique that is able to recover high degree-of-freedom articulated human body configurations in complex video sequences. We introduce the use of a novel mathematical technique, the product of exponential maps and twist motions, and its integration into a differential motion estimation. This results in solving simple linear systems, and enables us to recover robustly the kinematic degrees-offreedom in noise and complex self occluded configurations. We demonstrate this on several image sequences of people doing articulated full body movements, and visualize the results in re-animating an artificial 3D human model. We are also able to recover and re-animate the famous movements of Eadweard Muybridge's motion studies from the last century. To the best of our knowledge, this is the first computer vision based system that is able to process such challenging footage and recover complex motions with such high accuracy.

### Citations

8542 | Maximum likelihood from incomplete data via the EM algorithm
- Dempster, Laird, et al.
- 1977
(Show Context)
Citation Context ...including pixels from the environment. That distracts the exact motion measurement. Robust statistics would be one solution to this problem [4]. Another solution is an EM-based layered representation =-=[12, 15]-=-. It is beyond the scope of this paper to describe this method in detail, but we would like to outline the method briefly: We start with an initial guess of the support map (ellipsoidal projection in ... |

2090 | An iterative image registration technique with an application in stereo vision
- Lucas, Kanade
- 1981
(Show Context)
Citation Context ...1 ; a 2 ; a 3 ; a 4 ; d x ; d y ] T is defined as u(x; y; OE) = a 1 a 2 a 3 a 4 \Delta x y + d x d y (2) The first-order Taylor series expansion of (1) leads to the commonly used gradient formulation =-=[18]-=-: I t (x; y) + [I x (x; y); I y (x; y)] \Delta u(x; y; OE) = 0 (3) I t (x; y) is the temporal image gradient and [I x (x; y); I y (x; y)] is the spatial image gradient at location (x; y). Assuming a m... |

1552 | Good features to track
- Shi, Tomasi
- 1994
(Show Context)
Citation Context ...ation, section 3 details our experiments, and we discuss the results and future directions in section 4. 2 Motion Estimation We first describe a commonly used region-based motion estimation framework =-=[3, 25]-=-, and then describe the extension to kinematic chain constraints [20]. 2.1 Preliminaries Assuming that changes in image intensity are only due to translation of local image intensity, a parametric ima... |

1173 | Pfinder: Real-time tracking of the human body
- Wren, AZarbayejani, et al.
- 1997
(Show Context)
Citation Context ...etween the number of measurements and the degrees of freedom is even larger, because we can optimize over a complete image sequence. Alternative solutions to tracking of human bodies were proposed by =-=[27]-=- in tracking color blobs, and by [11] in using motion templates. Nonrigid models were proposed by [22, 7, 5, 6]. Section 2 introduces the new motion tracking framework and its mathematical formulation... |

663 | A mathematical introduction to robotic manipulation
- Murray, Sastry
- 1994
(Show Context)
Citation Context ... This reduces the number of free parameters dramatically. A convenient way of describing these additional domain constraints is the twist and product of exponential map formalism for kinematic chains =-=[20]-=-. The motion of one body segment can be described as the motion of the previous segment in a kinematic chain and an angular motion around a body joint. This adds just a single DOF for each additional ... |

594 | Hierarchical model-based motion estimation
- Bergen, Anandan, et al.
- 1992
(Show Context)
Citation Context ...em. Therefore all proposed motion estimation algorithms involve additional constraints about the assumed motion structure. One class of motion estimation techniques are based on parametric algorithms =-=[3]-=-. These techniques rely pm solving a highly overconstrained system of linear equations. For example, if an image patch could be modeled as a planar surface, an affine motion model with low degrees of ... |

547 |
The robust estimation of multiple motions: parametric and piecewise-smooth flow fields
- Black, Anandan
- 1996
(Show Context)
Citation Context ...e image. Such a support map usually covers a larger region, including pixels from the environment. That distracts the exact motion measurement. Robust statistics would be one solution to this problem =-=[4]-=-. Another solution is an EM-based layered representation [12, 15]. It is beyond the scope of this paper to describe this method in detail, but we would like to outline the method briefly: We start wit... |

236 | Tracking and Recognizing Rigid and Non-rigid Facial Motions Using Parametric Models of Image Motion
- Black, Yaccob
- 1995
(Show Context)
Citation Context ...ver a complete image sequence. Alternative solutions to tracking of human bodies were proposed by [27] in tracking color blobs, and by [11] in using motion templates. Nonrigid models were proposed by =-=[22, 7, 5, 6]-=-. Section 2 introduces the new motion tracking framework and its mathematical formulation, section 3 details our experiments, and we discuss the results and future directions in section 4. 2 Motion Es... |

212 | Model-based tracking of self-occluding articulated objects
- Rehg, Kanade
- 1995
(Show Context)
Citation Context ...ition, the exponential map formulation makes it possible to relate the image motion vectors linearly to the angular velocity. Others have modeled the humanbody with rigid segments connected at joints =-=[14, 24, 23, 13, 10, 9, 16, 17]-=-, but use different representations and features (for example DenavitHartenburg and edge detection). The introduction of twists and product of exponential maps into region-based motion estimation simp... |

204 | Cardboard people: A parameterized model of articulated image motion
- Ju, Black, et al.
- 1996
(Show Context)
Citation Context ...ition, the exponential map formulation makes it possible to relate the image motion vectors linearly to the angular velocity. Others have modeled the humanbody with rigid segments connected at joints =-=[14, 24, 23, 13, 10, 9, 16, 17]-=-, but use different representations and features (for example DenavitHartenburg and edge detection). The introduction of twists and product of exponential maps into region-based motion estimation simp... |

196 | Layered representation of motion video using robust maximumlikelihood estimation of mixture models and MDL encoding
- Ayer, Sawhney
- 1995
(Show Context)
Citation Context ...n be achieved. Problems occur if the motion of such a patch is not well described by the assumed motion model. Others have shown how to extend this approach to multipleindependent moving motion areas =-=[15, 1, 26]-=-. For each area, this approach still has the advantage that a large number of measurements are incorporated into a low DOF linear motion estimation. Problems occur if some of the areas do not have a l... |

188 |
Recovery of nonrigid motion and structure
- Pentland, Horowitz
- 1991
(Show Context)
Citation Context ...ver a complete image sequence. Alternative solutions to tracking of human bodies were proposed by [27] in tracking color blobs, and by [11] in using motion templates. Nonrigid models were proposed by =-=[22, 7, 5, 6]-=-. Section 2 introduces the new motion tracking framework and its mathematical formulation, section 3 details our experiments, and we discuss the results and future directions in section 4. 2 Motion Es... |

146 | Mixture models for optical flow computation
- Jepson, Black
- 1993
(Show Context)
Citation Context ...n be achieved. Problems occur if the motion of such a patch is not well described by the assumed motion model. Others have shown how to extend this approach to multipleindependent moving motion areas =-=[15, 1, 26]-=-. For each area, this approach still has the advantage that a large number of measurements are incorporated into a low DOF linear motion estimation. Problems occur if some of the areas do not have a l... |

122 | Motion regularization for model-based head tracking
- Basu, Essa, et al.
- 1996
(Show Context)
Citation Context ...jection.sEuler angles are commonly used to constrain the rotation matrix to SO(3), but they suffer from singularities and don't lead to a simple formulation in the optimization procedure (for example =-=[2]-=- propose a 3D ellipsoidal tracker based on Euler angles). In contrast, the twist representation provides a more elegant solution [20] and leads to a very simple linear representation of the motion mod... |

117 | Model-based estimation of 3d human motion with occlusion based on active multi-viewpoint selection
- Kakadiaris, Metaxas
- 1996
(Show Context)
Citation Context ...ition, the exponential map formulation makes it possible to relate the image motion vectors linearly to the angular velocity. Others have modeled the humanbody with rigid segments connected at joints =-=[14, 24, 23, 13, 10, 9, 16, 17]-=-, but use different representations and features (for example DenavitHartenburg and edge detection). The introduction of twists and product of exponential maps into region-based motion estimation simp... |

114 |
The representation and recognition of human movement using temporal templates
- Davis, Bobick
- 1997
(Show Context)
Citation Context ... the degrees of freedom is even larger, because we can optimize over a complete image sequence. Alternative solutions to tracking of human bodies were proposed by [27] in tracking color blobs, and by =-=[11]-=- in using motion templates. Nonrigid models were proposed by [22, 7, 5, 6]. Section 2 introduces the new motion tracking framework and its mathematical formulation, section 3 details our experiments, ... |

98 | Learning to track the visual motion of contours
- Blake, Isard, et al.
- 1995
(Show Context)
Citation Context ...ver a complete image sequence. Alternative solutions to tracking of human bodies were proposed by [27] in tracking color blobs, and by [11] in using motion templates. Nonrigid models were proposed by =-=[22, 7, 5, 6]-=-. Section 2 introduces the new motion tracking framework and its mathematical formulation, section 3 details our experiments, and we discuss the results and future directions in section 4. 2 Motion Es... |

89 | Towards 3-d model-based tracking and recognition of human movement: a multi-view approach,” on automatic face-and gesture-recognition
- Gavrila, Davis
- 1995
(Show Context)
Citation Context |

77 | Incremental recognition of pedestrians from image sequences
- Rohr
- 1993
(Show Context)
Citation Context |

71 | Learning parameterized models of image motion
- Black, Yacoob, et al.
- 1997
(Show Context)
Citation Context |

64 | Monocular tracking of the human arm in 3d
- Goncalves, Bernardo, et al.
- 1995
(Show Context)
Citation Context |

57 |
Walking Patterns of Normal Men
- Murray, Drought, et al.
- 1964
(Show Context)
Citation Context ...leg, based on more reliable measurements on the lower leg. We can compare the estimated angular configurations with motion capture data reported in the literature. Murray, Brought, and Kory published =-=[19]-=- such measurements for the hip, knee, and angle joints. We compared our motion tracker measurements with the published curves and found good agreement. Figure 3.1a shows the curves for the knee and an... |

30 | Perceptually organized EM: A framework for motion segmentaiton that combines information about form and motion
- Weiss, Adelson
- 1995
(Show Context)
Citation Context ...n be achieved. Problems occur if the motion of such a patch is not well described by the assumed motion model. Others have shown how to extend this approach to multipleindependent moving motion areas =-=[15, 1, 26]-=-. For each area, this approach still has the advantage that a large number of measurements are incorporated into a low DOF linear motion estimation. Problems occur if some of the areas do not have a l... |

21 | Video Motion Capture
- Bregler, Malik
- 1997
(Show Context)
Citation Context ... the kinematic model, and over the complete image sequence, instead of an image pair. Due to space limitations the details of the extended algorithm are described in the accompanying technical report =-=[8]-=-. 3 Results We applied this technique to video recordings in our lab and to photo-plate sequences of Eadweard Muybdrige's motion studies [21]. Figure 2: Example configurations of the estimated kinemat... |

13 |
A program to see a walking person
- Hogg
- 1983
(Show Context)
Citation Context |

7 |
Automatic face and gestual recognition for video indexing
- Clergue, Goldber, et al.
- 1995
(Show Context)
Citation Context |

5 |
The Human Figure In Motion. Various Publishers, latest edition by Dover
- Muybridge
- 1901
(Show Context)
Citation Context ...lgorithm are described in the accompanying technical report [8]. 3 Results We applied this technique to video recordings in our lab and to photo-plate sequences of Eadweard Muybdrige's motion studies =-=[21]-=-. Figure 2: Example configurations of the estimated kinematic structure. First image shows the support maps of the initial configuration. In subsequent images the white lines show blob axes. The joint... |

2 |
and B.Horowitz. Recoveryof nonrigid motion and structure
- Pentland
- 1991
(Show Context)
Citation Context |