## 3D hand tracking by rapid stochastic gradient descent using a skinning model (2004)

### Cached

### Download Links

Venue: | IN 1ST EUROPEAN CONFERENCE ON VISUAL MEDIA PRODUCTION (CVMP |

Citations: | 21 - 3 self |

### BibTeX

@INPROCEEDINGS{Bray043dhand,

author = {Matthieu Bray and Esther Koller-Meier and Pascal Müller and Luc Van Gool and Nicol N. Schraudolph},

title = {3D hand tracking by rapid stochastic gradient descent using a skinning model},

booktitle = {IN 1ST EUROPEAN CONFERENCE ON VISUAL MEDIA PRODUCTION (CVMP},

year = {2004},

pages = {59--68},

publisher = {}

}

### OpenURL

### Abstract

The main challenge of tracking articulated structures like hands is their large number of degrees of freedom (DOFs). A realistic 3D model of the human hand has at least 26 DOFs. The arsenal of tracking approaches that can track such structures fast and reliably is still very small. This paper proposes a tracker based on ‘Stochastic Meta-Descent ’ (SMD) for optimizations in such highdimensional state spaces. This new algorithm is based on a gradient descent approach with adaptive and parameter-specific step sizes. The SMD tracker facilitates the integration of constraints, and combined with a stochastic sampling technique, can get out of spurious local minima. Furthermore, the integration of a deformable hand model based on linear blend skinning and anthropometrical measurements reinforce the robustness of our tracker. Experiments show the efficiency of the SMD algorithm in comparison with common optimization methods.

### Citations

1131 | CONDENSATION conditional density propagation for visual tracking
- Isard, Blake
- 1998
(Show Context)
Citation Context ...h common optimization methods. 1 INTRODUCTION Relatively little work has been devoted to the tracking of articulated structures [1, 2, 3, 4, 5]. Although the Condensation algorithm by Isard and Blake =-=[6]-=- has shown to be a robust and powerful stochastic sampling approach, it is not really applicable to this kind of tracking application. A dense sampling is needed to guarantee a good solution, but this... |

750 |
An algorithm for least-squares estimation of nonlinear parameters
- MARQUARDT
- 1963
(Show Context)
Citation Context ...minima but is computationally expensive. When derivatives are available, nonlinear optimization problems are often solved by second-order gradient techniques such as the Levenberg-Marquardt algorithm =-=[19, 20]-=- or truncated quasi-Newton method like the BroydenFletcher-Goldfarb-Shanno (BFGS) algorithm [18]. The basic idea of these methods is to build up, iteratively an approximation to the inverse of the Hes... |

401 | Articulated Body Motion Capture by Annealed Particle Filtering
- Deutscher, Blake, et al.
- 2000
(Show Context)
Citation Context .... Experiments show the efficiency of the SMD algorithm in comparison with common optimization methods. 1 INTRODUCTION Relatively little work has been devoted to the tracking of articulated structures =-=[1, 2, 3, 4, 5]-=-. Although the Condensation algorithm by Isard and Blake [6] has shown to be a robust and powerful stochastic sampling approach, it is not really applicable to this kind of tracking application. A den... |

339 |
Increased rates of convergence through learning rate adaptat ion
- Jacobs
- 1988
(Show Context)
Citation Context ... step sizes is in effect a diagonal conditioner for the gradient system. By observing the sign of the autocorrelation of the gradient, one obtains simple strategies for adapting a during optimization =-=[21, 22, 23]-=-: the step size is decreased along directions where the gradient oscillates (i.e., g t·g t−1<0), and increased along those where it does not. This approach should not be used in combination with stoch... |

317 | Stochastic tracking of 3D human figures using 2D image motion
- Sidenbladh, Black, et al.
- 2000
(Show Context)
Citation Context .... Experiments show the efficiency of the SMD algorithm in comparison with common optimization methods. 1 INTRODUCTION Relatively little work has been devoted to the tracking of articulated structures =-=[1, 2, 3, 4, 5]-=-. Although the Condensation algorithm by Isard and Blake [6] has shown to be a robust and powerful stochastic sampling approach, it is not really applicable to this kind of tracking application. A den... |

205 | Model-based tracking of self-occluding articulated objects
- Rehg, Kanade
- 1995
(Show Context)
Citation Context .... Experiments show the efficiency of the SMD algorithm in comparison with common optimization methods. 1 INTRODUCTION Relatively little work has been devoted to the tracking of articulated structures =-=[1, 2, 3, 4, 5]-=-. Although the Condensation algorithm by Isard and Blake [6] has shown to be a robust and powerful stochastic sampling approach, it is not really applicable to this kind of tracking application. A den... |

203 | N.: Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation
- LEWIS, CORDNER, et al.
(Show Context)
Citation Context ...D modeling and animation software packages and they go by many names. MagnenatThalmann et al. called it ’Joint-dependent Local Deformation’ [9], Lewis et al. called it ’Skeleton Subspace Deformation’ =-=[11]-=-, Alias’ Maya calls it ’Smooth Skinning’ 1 and in one of the most recent publications in this field, it is called ’Linear Blend Skinning’ [12]. First, a hierarchical skeleton is placed inside a static... |

158 | Real-time 3D model acquisition
- RUSINKIEWICZ, HALL-HOLT, et al.
- 2002
(Show Context)
Citation Context ...oduced at this speed. Hence, the 3D results are provided at a rate that is too low to support on-line tracking. This said, alternative structured light systems exist that do reach the necessary speed =-=[16, 17]-=-. The transition to the system described in [16] is in preparation at the time of writing. In order not to fall victim to erroneous 3D data, which tend to cluster near the rim of the hand, we apply sk... |

134 |
Additive versus exponentiated gradient updates for linear prediction
- Kivinen, Warmuth
- 1997
(Show Context)
Citation Context ...he notion of adapting a by a meta-level gradient descent in E as well [21]. To allow a to cover a wide dynamic range while remaining strictly positive, the meta-level descent is best performed on lna =-=[25]-=-. Assuming that each element of a affects the objective E only through the corresponding element of p, such that (d) (c) t0 p(t) a(t) p(t) a(t) Figure 5: Dependence of a parameter p on its step size a... |

131 | Covariance scaled sampling for monocular 3d body tracking
- Sminchisescu, Triggs
- 2001
(Show Context)
Citation Context |

107 | M.: Building efficient, accurate character skins from examples
- MOHR, GLEICHER
(Show Context)
Citation Context ...], Lewis et al. called it ’Skeleton Subspace Deformation’ [11], Alias’ Maya calls it ’Smooth Skinning’ 1 and in one of the most recent publications in this field, it is called ’Linear Blend Skinning’ =-=[12]-=-. First, a hierarchical skeleton is placed inside a static polygon-model. This initial pose has, again, various names: binding pose, dress pose etc. The difference from a usual polygon model is that e... |

75 | Jointdependent local deformations for hand animation and object grasping,” Graphics Interface
- Magnenat-Thalmann, Laperrire, et al.
- 1988
(Show Context)
Citation Context ...ing by producing more subtle effects like the deformation of the palm. Hierarchically structured character skin deformation was introduced by Catmull [8]. In subsequent work, Magnenat-Thalmann et al. =-=[9]-=- and Komatsu [10] went beyond rigid skinning techniques and demonstrated human body deformation driven by an underlying skeleton. They developed custom programmed algorithms for each joint. We have ch... |

75 | Adapting bias by gradient descent: an incremental version of delta-bar-delta
- Sutton
- 1992
(Show Context)
Citation Context ...verage of past gradients [21]. Although such ad-hoc smoothing does improve performance, it does not model long-term dependencies, the average being one of immediate, single-step effects. By contrast, =-=[26, 27]-=- models the long-term effect of a on future parameter values in a linear system by carrying thesrelevant partials forward through time. This results in an iterative update rule for v extended to non-l... |

69 | Fast exact multiplication by the Hessian
- Pearlmutter
- 1994
(Show Context)
Citation Context ... that SMD is a computationally expensive algorithm. Fortunately this is not the case, since there are very efficient indirect methods for computing the product of the Hessian with an arbitrary vector =-=[28]-=-. To prevent negative eigenvalues from causing (14) to diverge, SMD uses an extended Gauss-Newton approximation that also admits a fast matrix-vector product [7]. In our case, this is given by Htvt ≈ ... |

63 |
Supersab: fast adaptive backpropagation with good scaling properties
- tollenaere
- 1990
(Show Context)
Citation Context ... step sizes is in effect a diagonal conditioner for the gradient system. By observing the sign of the autocorrelation of the gradient, one obtains simple strategies for adapting a during optimization =-=[21, 22, 23]-=-: the step size is decreased along directions where the gradient oscillates (i.e., g t·g t−1<0), and increased along those where it does not. This approach should not be used in combination with stoch... |

42 |
Speeding up backpropagation
- Silva, Almeida
- 1990
(Show Context)
Citation Context ... step sizes is in effect a diagonal conditioner for the gradient system. By observing the sign of the autocorrelation of the gradient, one obtains simple strategies for adapting a during optimization =-=[21, 22, 23]-=-: the step size is decreased along directions where the gradient oscillates (i.e., g t·g t−1<0), and increased along those where it does not. This approach should not be used in combination with stoch... |

42 | Gain adaptation beats least squares
- Sutton
- 1992
(Show Context)
Citation Context ...verage of past gradients [21]. Although such ad-hoc smoothing does improve performance, it does not model long-term dependencies, the average being one of immediate, single-step effects. By contrast, =-=[26, 27]-=- models the long-term effect of a on future parameter values in a linear system by carrying thesrelevant partials forward through time. This results in an iterative update rule for v extended to non-l... |

38 | Fast curvature matrix-vector products for second-order gradient descent
- Schraudolph
- 2002
(Show Context)
Citation Context ...ime and between dimensions. In each iteration, the appropriate changes are automatically determined by a meta-level gradient descent. Individual parts of the SMD algorithm have already been published =-=[7]-=- but to the best of our knowledge not the overall method. Furthermore, they have been only applied to neural networks so far. The integration to a visual hand tracking framework needs modifications si... |

34 | RealTime Range Scanning of Deformable Surfaces by Adaptively Coded Structured Light,” 3DIM
- Koninckx, Griesser, et al.
- 2003
(Show Context)
Citation Context ...oduced at this speed. Hence, the 3D results are provided at a rate that is too low to support on-line tracking. This said, alternative structured light systems exist that do reach the necessary speed =-=[16, 17]-=-. The transition to the system described in [16] is in preparation at the time of writing. In order not to fall victim to erroneous 3D data, which tend to cluster near the rim of the hand, we apply sk... |

31 |
Human Skin Model Capable of Natural Shape Variation
- Komatsu
- 1988
(Show Context)
Citation Context ... more subtle effects like the deformation of the palm. Hierarchically structured character skin deformation was introduced by Catmull [8]. In subsequent work, Magnenat-Thalmann et al. [9] and Komatsu =-=[10]-=- went beyond rigid skinning techniques and demonstrated human body deformation driven by an underlying skeleton. They developed custom programmed algorithms for each joint. We have chosen this non-phy... |

20 | Multi-player residual advantage learning with general function approximation (Tech
- Harmon, Baird
- 1996
(Show Context)
Citation Context ... we find that vt+1 ≡ − ∂ p t+1 ∂ lnat t0 = at · g t (a) (b) (11) which, inserted into (10), gives us the familiar autocorrelation g t ·g t−1 of the gradient, now without the problematic sign function =-=[32, 24]-=-. Finally, exponentiating (10) gives at = at−1·exp(µ gt·vt) ≈ at−1·max( 1 2 , 1+µvt·g t) (12) The linearization eu ≈ max( 1 2 , 1 + u) eliminates the expensive exponentiation for each weight update, w... |

13 |
A system for computer generated movies
- Catmull
- 1972
(Show Context)
Citation Context ... hand shapes, and hence also better supports tracking by producing more subtle effects like the deformation of the palm. Hierarchically structured character skin deformation was introduced by Catmull =-=[8]-=-. In subsequent work, Magnenat-Thalmann et al. [9] and Komatsu [10] went beyond rigid skinning techniques and demonstrated human body deformation driven by an underlying skeleton. They developed custo... |

10 |
A Method for the Solution of Certain Non-Linear
- Levenberg
- 1944
(Show Context)
Citation Context ...minima but is computationally expensive. When derivatives are available, nonlinear optimization problems are often solved by second-order gradient techniques such as the Levenberg-Marquardt algorithm =-=[19, 20]-=- or truncated quasi-Newton method like the BroydenFletcher-Goldfarb-Shanno (BFGS) algorithm [18]. The basic idea of these methods is to build up, iteratively an approximation to the inverse of the Hes... |

6 |
Inkwell: A 2.5D Animation System
- Litwinowicz
- 1991
(Show Context)
Citation Context ...econd). Therefore, the tracking results need to be cleaned up in order to use them for animation purposes. There is an extensive literature about noise reduction integrated into motion capture system =-=[29, 30, 31]-=-. We chose to use a simple linear adaptive filter which substitutes each data value by a linear combination of its neighboors [29]. Figure 8: Erroneous depth map from the sequence presented in Figure ... |

1 |
Anthropometric Data for Describing the
- Buchholz, Armstrong, et al.
- 1992
(Show Context)
Citation Context ...we have chosen to consider them as DOFs, moving in a range of ± 4 degrees. The hard constraints and dependencies of the other angles are the same as used in [14]. The anthropometrical measurements in =-=[13, 15]-=- give very good approximations for joint-positions. In these two papers, statistical relationships between the skin and the underlying skeleton have been established. Figure 1 shows the resulting hand... |

1 |
An Integrated Approach towards the Representation
- Sudarsky, House
- 2000
(Show Context)
Citation Context ...econd). Therefore, the tracking results need to be cleaned up in order to use them for animation purposes. There is an extensive literature about noise reduction integrated into motion capture system =-=[29, 30, 31]-=-. We chose to use a simple linear adaptive filter which substitutes each data value by a linear combination of its neighboors [29]. Figure 8: Erroneous depth map from the sequence presented in Figure ... |