## Robust 3D head tracking using camera pose estimation (2006)

### Cached

### Download Links

- [www.cs.technion.ac.il]
- [www.cs.technion.ac.il]
- DBLP

### Other Repositories/Bibliography

Venue: | In 18th International Conference on Pattern Recognition (ICPR |

Citations: | 6 - 0 self |

### BibTeX

@INPROCEEDINGS{Ohayon06robust3d,

author = {Shay Ohayon and Ehud Rivlin},

title = {Robust 3D head tracking using camera pose estimation},

booktitle = {In 18th International Conference on Pattern Recognition (ICPR},

year = {2006},

pages = {1063--1066}

}

### OpenURL

### Abstract

In this paper a robust method to track a head in 3D using a static monocular camera is presented. Head pose is recovered by formulating the problem as a camera pose estimation problem. Several 3D feature points are acquired from the head prior to tracking and used as a model. Both artificial and natural occurring features can be used. Pose is estimated by solving a robust version of ”Perspective n Point ” problem (PnP). The proposed algorithm can handle self occlusions, outliers and recover from tracking failures. Results were validated by simulations and were compared to pose obtained using an accurate magnetic field 3D measuring device. Our system is not limited to tracking human heads and can be used to track animal heads as well. To demonstrate the applicability of our method, three types of heads were tracked (human, barn owl, chameleon) in a series of biological experiments. 1.

### Citations

5109 | Distinctive image features from scale-invariant keypoints
- Lowe
(Show Context)
Citation Context ...ther circular or rectangular in shape and had a distinct spatial characteristics, which made them easy to detect. The second type of features we used were natural occurring. In a recent study by Lowe =-=[14]-=-, it was shown that reliable natural occurring features, which are invariant under scaling, rotation and limited view change, can be extracted from images. Features are local extremum points in scale ... |

3158 |
A.: “Multiple View Geometry in Computer Vision
- Hartley, Zisserman
- 2004
(Show Context)
Citation Context ... body object and is not limited to human heads as will be shown subsequently. The problem of external camera pose estimation relative to a 3D set of reference points was addressed by many researchers =-=[9, 5, 13, 7, 19]-=-. Most of the proposed methods rely on knowledge of absolute distances from camera center to features. The ”Perspective n Point” (PnP) problem ([19]) is an exception, since it uses the relative distan... |

2452 |
Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography
- Fischler, Bolles
- 1981
(Show Context)
Citation Context ... body object and is not limited to human heads as will be shown subsequently. The problem of external camera pose estimation relative to a 3D set of reference points was addressed by many researchers =-=[9, 5, 13, 7, 19]-=-. Most of the proposed methods rely on knowledge of absolute distances from camera center to features. The ”Perspective n Point” (PnP) problem ([19]) is an exception, since it uses the relative distan... |

831 | A flexible new technique for camera calibration
- Zhang
(Show Context)
Citation Context ...quired model (2) �P u R − P v R� = �P u M − P v M � = du,v. (3)sWe assume that camera internal parameters (focal length, skew, deviation from optical axis) are known and represented as a 3x3 matrix K =-=[24]-=-. The direction vector, from camera center C to feature u is given by: N(u) ∆ = K −1 [pu, 1] T . (4) Therefore, the angle ∠P u RCP v R can be found from the two direction vectors N(u) and N(v): cos ∠P... |

719 | Closed-form solution of absolute orientation using unit quaternions
- HORN
- 1987
(Show Context)
Citation Context ...Orientation problem. The least squares formulation, which minimizes this alignment error is min R,T � u �P u R − (RP u M + T )� 2 . (12) Linear closed form solutions to this problem were suggested by =-=[10, 6]-=-. The linear methods which solve the absolute orientation problem are not optimal in our case. A more accurate solution can be obtained by projecting the 3D recovered model to the image and minimizing... |

419 |
Least-squares fitting of two 3-d point sets
- Arun, Huang, et al.
(Show Context)
Citation Context ...computer vision and is called Absolute Orientation problem. The minimum square formulation is given by argmin R,T M� u=1 [P u R − (RP u M + T )] 2 . (24) Several solutions are known ([10], [15],[16], =-=[19]-=-). We have selected the method of Faugeras, who found a close form solution using quaternions. Let us first assume that the two models are centered about their centroid. In that case, the problem is r... |

360 |
Animating Rotation with Quaternion Curves
- Shoemake
(Show Context)
Citation Context ...min (q ∗ P q,s.t.�q�=1 u=1 u M − P u R M� M� ∗ q)2 , (25) where ¯q represents the quaternion conjugate and ∗ denotes quaternion multiplication (for more details on quaternion algebra, please refer to =-=[32]-=-). The term q ∗ P u M − P u R argmin q,s.t.�q�=1 u=1 Differentiation by q results in: M� (Aiq) 2 = argmin = argmin q q,s.t.�q�=1 T B4x4q ∗ q is linear in q and can be written as: M� q q,s.t.�q�=1 u=1 ... |

288 | Fitting parameterized three-dimensional models to images
- Lowe
- 1991
(Show Context)
Citation Context ... body object and is not limited to human heads as will be shown subsequently. The problem of external camera pose estimation relative to a 3D set of reference points was addressed by many researchers =-=[9, 5, 13, 7, 19]-=-. Most of the proposed methods rely on knowledge of absolute distances from camera center to features. The ”Perspective n Point” (PnP) problem ([19]) is an exception, since it uses the relative distan... |

259 | Elliptical Head Tracking Using Intensity Gradients and Colour Histograms
- Birchfield
- 1998
(Show Context)
Citation Context ...lem of tracking a human head has received much attention in past years. Algorithms differ by their representations of the head and its movement. While simple and fast methods track an ellipse in 2D ( =-=[1, 16]-=- ), they are unable to report position in 3D, nor orientation. In [20, 4] partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low. Accurate 3D ... |

229 |
The representation, recognition, and locating of 3-d objects
- Faugeras, Hebert
- 1986
(Show Context)
Citation Context ...Orientation problem. The least squares formulation, which minimizes this alignment error is min R,T � u �P u R − (RP u M + T )� 2 . (12) Linear closed form solutions to this problem were suggested by =-=[10, 6]-=-. The linear methods which solve the absolute orientation problem are not optimal in our case. A more accurate solution can be obtained by projecting the 3D recovered model to the image and minimizing... |

201 | Model-based object pose in 25 lines of code
- Dementhon, Davis
- 1995
(Show Context)
Citation Context |

191 |
Analysis and solutions of the three point perspective pose estimation problem
- Haralick, Lee, et al.
- 1991
(Show Context)
Citation Context ...ree quadratic equations in three unknowns (�P u R � , �P v R � , �P w R �) are obtained. Several algebraic methods are known to reduce the set of three equations into a single equation in one unknown =-=[8]-=-. These methods yield a forth degree polynomial: such that: g (x) ≡ 4� ai(x) i , (6) i=0 g (xu) = 0, xu ≡ �P u R� 2 . (7) In [8], Haralick et al review six reduction methods and analyze their numerica... |

103 | Three-dimensional face recognition
- Bronstein, Bronstein, et al.
- 2004
(Show Context)
Citation Context ...tative image of the head. These features represented the acquired model. The 3D position of all features marked in yellow were estimated using structured light technique, which is described elsewhere =-=[1]-=-. In each new video frame, SIFT features were matched against the acquired model. 11sTechnion - Computer Science Department - Technical Report CS-2006-12 - 2006 Figure 4: The natural occurring feature... |

76 | Monocular model-based 3d tracking of rigid objects: A survey
- Lepetit, Fua
(Show Context)
Citation Context ... those systems is low. Accurate 3D head tracking systems use various models to represent the head as a rigid body, which moves in 3D space. An excellent survey of model based tracking can be found at =-=[12]-=-. Previous models represented the head as a triangular mesh [22], a textured cylinder [3, 2] )or superquadric ([15, 23]). While a highly detailed representation gives better accuracy in pose estimatio... |

56 | Synergistic Face Detection and Pose Estimation with Energy-Based Models
- Osadchy, LeCun, et al.
- 2007
(Show Context)
Citation Context ...nted in Real-Time. However, light changes and different color skins result in tracking failures. Another drawback with such approaches is the inability to report head orientation. In [31], [7], [20], =-=[28]-=-, partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low (up to 15 degree error in estimating rotation). More accurate systems use 3D geometri... |

54 | Active face tracking and pose estimation in an interactive room
- Darrell, Moghaddam, et al.
- 1996
(Show Context)
Citation Context ...lgorithms differ by their representations of the head and its movement. While simple and fast methods track an ellipse in 2D ( [1, 16] ), they are unable to report position in 3D, nor orientation. In =-=[20, 4]-=- partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low. Accurate 3D head tracking systems use various models to represent the head as a rigid... |

50 | Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes,” Visual Cognition 12
- Itti
- 2005
(Show Context)
Citation Context ...sarily human), as will be shown subsequently. There is a growing increase of interest in gaze tracking systems from many areas, such as movement analysis [24], behavior analysis [13], [18], attention =-=[17]-=- and more. Although the main thrust of this paper is to present a head tracking algorithm, a full gaze tracking system can be implemented easily, since knowledge about head position simplifies the eye... |

47 | Deriving action and behavior primitives from human motion data
- Jenkins, Mataric
- 2002
(Show Context)
Citation Context ...metry (not necessarily human), as will be shown subsequently. There is a growing increase of interest in gaze tracking systems from many areas, such as movement analysis [24], behavior analysis [13], =-=[18]-=-, attention [17] and more. Although the main thrust of this paper is to present a head tracking algorithm, a full gaze tracking system can be implemented easily, since knowledge about head position si... |

27 |
Vision-based pose computation: Robust and accurate augmented reality tracking
- Park, Jiang, et al.
- 1999
(Show Context)
Citation Context ...itive solution is not guaranteed since it is computed from the eigenvector of the coefficient matrix. Another forthcoming in using the SVD method is instability when outliers are present. Inspired by =-=[18]-=-, we use robust M-estimator ρ, such as Huber, to overcome the above difficulties. A robust estimation of xu is obtained by minimizing xu = arg min λ� � 4� ρ j=1 i=0 a j i xi � . (9) This problem is no... |

22 | Head tracking via robust registration in texture map images
- Cascia, Isidoro, et al.
- 1998
(Show Context)
Citation Context ...he head as a rigid body, which moves in 3D space. An excellent survey of model based tracking can be found at [12]. Previous models represented the head as a triangular mesh [22], a textured cylinder =-=[3, 2]-=- )or superquadric ([15, 23]). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition. In [11], Hannuksela propos... |

20 |
The Perspective View of Three Points
- Wolfe, Mathis, et al.
- 1991
(Show Context)
Citation Context ...ever, this polynomial can have up to four different real roots. Thus, three features are not enough to obtain a unique solution. A geometrical interpretation of the multiple solutions can be found at =-=[21]-=-. Four points are sufficient for an unambiguous solution, unless they lie in a degenerate configuration (C, P u R , P v R , P w R are on a plane). When N ≥ 4 points are available, then λ ≡ � � N−1 2 f... |

20 | Model-based head pose tracking with stereovision
- Yang, Zhang
- 2002
(Show Context)
Citation Context ...ious models to represent the head as a rigid body, which moves in 3D space. An excellent survey of model based tracking can be found at [12]. Previous models represented the head as a triangular mesh =-=[22]-=-, a textured cylinder [3, 2] )or superquadric ([15, 23]). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition... |

17 |
3D head tracking using motion adaptive texture-mapping
- Brown
- 2001
(Show Context)
Citation Context ...he head as a rigid body, which moves in 3D space. An excellent survey of model based tracking can be found at [12]. Previous models represented the head as a triangular mesh [22], a textured cylinder =-=[3, 2]-=- )or superquadric ([15, 23]). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition. In [11], Hannuksela propos... |

17 | 3D head tracking under partial occlusion
- Zhang, Kambhamettu
(Show Context)
Citation Context ...which moves in 3D space. An excellent survey of model based tracking can be found at [12]. Previous models represented the head as a triangular mesh [22], a textured cylinder [3, 2] )or superquadric (=-=[15, 23]-=-). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition. In [11], Hannuksela proposed tosuse the Pose from 3 P... |

17 |
3D-tracking of head and hands for pointing gesture recognition in a human-robot interaction scenario
- Nickel, Seemann, et al.
- 2004
(Show Context)
Citation Context ...ain 3D information, which is matched against the known model. A major shortcoming of this method is the amount of time one need to spend to create a precise model. Recent approaches, such as [6] [5], =-=[26]-=-, use a cylinder to approximate both head underlying geometry and texture. A linear combination of motion adaptive templates is used to match the model against current image. Since a cylinder is only ... |

15 | A Robust Model-Based Approach for 3D Head Tracking in Video Sequences
- Malciu, Preteux
(Show Context)
Citation Context ...which moves in 3D space. An excellent survey of model based tracking can be found at [12]. Previous models represented the head as a triangular mesh [22], a textured cylinder [3, 2] )or superquadric (=-=[15, 23]-=-). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition. In [11], Hannuksela proposed tosuse the Pose from 3 P... |

13 |
N-Point Camera Pose Determination
- Quan, Lan, et al.
- 1999
(Show Context)
Citation Context |

13 | Camera pose revisited: New linear algorithms
- Ameller, Triggs, et al.
- 2002
(Show Context)
Citation Context ...ints, although four are needed to resolve ambiguities [33]. Recently, new linear algorithms were developed for the four point problem (P4P), that do not depend on a complex coefficient representation =-=[2]-=-. However, error analysis showed that they have larger error than the basic P3P algorithms. In principle, any solution to the camera pose estimation algorithm that uses only the relative distances, ca... |

9 | Model-based head tracking and 3D pose estimation
- Prêteux, Malciu
- 1998
(Show Context)
Citation Context ...d translations. Brolly et al [4] use Nurbs surface with texture to synthesize both appearance and pose, but could not report pose accuracy since ground truth was unavailable. Several researches [23], =-=[9]-=-, [35] introduced the notion of extended superquadric surface, or fourier synthesized representation of a surface, which has high degree of flexibility to encompass face structure. They use model indu... |

9 |
Behavior Classification by Eigendecomposition of Periodic Motions
- Goldenberg, Kimmel, et al.
- 2005
(Show Context)
Citation Context ...ry geometry (not necessarily human), as will be shown subsequently. There is a growing increase of interest in gaze tracking systems from many areas, such as movement analysis [24], behavior analysis =-=[13]-=-, [18], attention [17] and more. Although the main thrust of this paper is to present a head tracking algorithm, a full gaze tracking system can be implemented easily, since knowledge about head posit... |

8 | Head Tracking Using Stereo
- Russakoff, Herman
- 2002
(Show Context)
Citation Context ...lgorithms differ by their representations of the head and its movement. While simple and fast methods track an ellipse in 2D ( [1, 16] ), they are unable to report position in 3D, nor orientation. In =-=[20, 4]-=- partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low. Accurate 3D head tracking systems use various models to represent the head as a rigid... |

7 | Model-based head pose estimation for air-traffic controllers
- Brolly, Stratelos, et al.
- 2003
(Show Context)
Citation Context ...nly a rough approximation to head geometry, those methods suffer from inaccuracies in estimating rotation, and have difficulties differentiating between small rotations and translations. Brolly et al =-=[4]-=- use Nurbs surface with texture to synthesize both appearance and pose, but could not report pose accuracy since ground truth was unavailable. Several researches [23], [9], [35] introduced the notion ... |

4 |
Detecting and tracking human faces in videos
- Li, Goshtasby, et al.
(Show Context)
Citation Context ...mplemented in Real-Time. However, light changes and different color skins result in tracking failures. Another drawback with such approaches is the inability to report head orientation. In [31], [7], =-=[20]-=-, [28], partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low (up to 15 degree error in estimating rotation). More accurate systems use 3D ge... |

3 | 3d real-time head tracking fusing color histograms and stereovision
- Moreno, Tarrida, et al.
- 2002
(Show Context)
Citation Context ...lem of tracking a human head has received much attention in past years. Algorithms differ by their representations of the head and its movement. While simple and fast methods track an ellipse in 2D ( =-=[1, 16]-=- ), they are unable to report position in 3D, nor orientation. In [20, 4] partial orientation information, such as tilt or yaw is available. However, the accuracy of those systems is low. Accurate 3D ... |

2 | A real-time facial feature based head tracker
- Hannuksela, Pietikainen
(Show Context)
Citation Context ...extured cylinder [3, 2] )or superquadric ([15, 23]). While a highly detailed representation gives better accuracy in pose estimation, it suffers from a time consuming process of model acquisition. In =-=[11]-=-, Hannuksela proposed tosuse the Pose from 3 Points algorithm to solve for head pose. However, their algorithm can not handle self occlusions and will fail when one of the three tracked points is obsc... |

2 |
Off-centric rotation axes in natural head movements: implications for vestibular reaerence and kinematic redundancy. Journal of Neurophysiology
- Medendorp, Gisbergen, et al.
(Show Context)
Citation Context ...ing of head with arbitrary geometry (not necessarily human), as will be shown subsequently. There is a growing increase of interest in gaze tracking systems from many areas, such as movement analysis =-=[24]-=-, behavior analysis [13], [18], attention [17] and more. Although the main thrust of this paper is to present a head tracking algorithm, a full gaze tracking system can be implemented easily, since kn... |

1 |
On the nature of barn owls pre-attack head movments. Submitted to
- Ohayon
- 2005
(Show Context)
Citation Context ...strange head movements prior to prey capture while they sit on a perch and observe their prey. These pre-attack head movements were successfully tracked and analyzed with our system in a recent study =-=[17]-=-. The observed pre-attack movements are believed to have a functional role in prey detection or distance estimation. High pose accuracy was obtained with the usage of artificial markers. Several circu... |