Results 1  10
of
31
Twin Gaussian Processes for Structured Prediction
, 2010
"... ... generic structured prediction method that uses Gaussian process (GP) priors on both covariates and responses, both multivariate, and estimates outputs by minimizing the KullbackLeibler divergence between two GP modeled as normal distributions over finite index sets of training and testing examp ..."
Abstract

Cited by 62 (4 self)
 Add to MetaCart
... generic structured prediction method that uses Gaussian process (GP) priors on both covariates and responses, both multivariate, and estimates outputs by minimizing the KullbackLeibler divergence between two GP modeled as normal distributions over finite index sets of training and testing examples, emphasizing the goal that similar inputs should produce similar percepts and this should hold, on average, between their marginal distributions. TGP captures not only the interdependencies between covariates, as in a typical GP, but also those between responses, so correlations among both inputs and outputs are accounted for. TGP is exemplified, with promising results, for the reconstruction of 3d human poses from monocular and multicamera video sequences in the recently introduced HumanEva benchmark, where we achieve 5 cm error on average per 3d marker for models trained jointly, using data from multiple people and multiple activities. The method is fast and automatic: it requires no handcrafting of the initial pose, camera calibration parameters, or the availability of a 3d body model associated with human subjects used for training or testing.
PhysicsBased Person Tracking Using the Anthropomorphic Walker
, 2010
"... We introduce a physicsbased model for 3D person tracking. Based on a biomechanical characterization of lowerbody dynamics, the model captures important physical properties of bipedal locomotion such as balance and ground contact. The model generalizes naturally to variations in style due to change ..."
Abstract

Cited by 29 (2 self)
 Add to MetaCart
(Show Context)
We introduce a physicsbased model for 3D person tracking. Based on a biomechanical characterization of lowerbody dynamics, the model captures important physical properties of bipedal locomotion such as balance and ground contact. The model generalizes naturally to variations in style due to changes in speed, steplength, and mass, and avoids common problems (such as footskate) that arise with existing trackers. The dynamics comprise a two degreeoffreedom representation of human locomotion with inelastic ground contact. A stochastic controller generates impulsive forces during the toeoff stage of walking, and springlike forces between the legs. A higherdimensional kinematic body model is conditioned on the underlying dynamics. The combined model is used to track walking people in video, including examples with turning, occlusion, and varying gait. We also report quantitative monocular and binocular tracking results with the HumanEva dataset.
Videomocap: modeling physically realistic human motion from monocular video sequences
 ACM Transactions on Graphics
"... Figure 1: Modeling physically realistic human motion from uncalibrated monocular video sequences. This paper presents a videobased motion modeling technique for generating physically realistic human motion from monocular video sequences. We formulate the videobased motion modeling process in an im ..."
Abstract

Cited by 23 (2 self)
 Add to MetaCart
Figure 1: Modeling physically realistic human motion from uncalibrated monocular video sequences. This paper presents a videobased motion modeling technique for generating physically realistic human motion from monocular video sequences. We formulate the videobased motion modeling process in an imagebased keyframe animation framework. The system first computes camera parameters, human skeletal size, and a small number of 3D key poses from video and then uses 2D image measurements at intermediate frames to automatically calculate the “in between ” poses. During reconstruction, we leverage Newtonian physics, contact constraints, and 2D image measurements to simultaneously reconstruct fullbody poses, joint torques, and contact forces. We have demonstrated the power and effectiveness of our system by generating a wide variety of physically realistic human actions from uncalibrated monocular video sequences such as sports video footage.
C.: Latent Structured Models for Human Pose Estimation. ICCV
, 2011
"... We present an approach for automatic 3D human pose reconstruction from monocular images, based on a discriminative formulation with latent segmentation inputs. We advance the field of structured prediction and human pose reconstruction on several fronts. First, by working with a pool of figuregroun ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
(Show Context)
We present an approach for automatic 3D human pose reconstruction from monocular images, based on a discriminative formulation with latent segmentation inputs. We advance the field of structured prediction and human pose reconstruction on several fronts. First, by working with a pool of figureground segment hypotheses, the prediction problem is formulated in terms of combined learning and inference over segment hypotheses and 3D human articular configurations. Besides constructing tractable formulations for the combined segment selection and pose estimation problem, we propose new augmented kernels that can better encode complex dependencies between output variables. Furthermore, we provide primal linear reformulations based on Fourier kernel approximations, in order to scaleup the nonlinear latent structured prediction methodology. The proposed models are shown to be competitive in the HumanEva benchmark and are also illustrated in a clip collected from a Hollywood movie, where the model can infer human poses from monocular images captured in complex environments. 1.
Estimating Contact Dynamics
"... Motion and interaction with the environment are fundamentally intertwined. Few peopletracking algorithms exploit such interactions, and those that do assume that surface geometry and dynamics are given. This paper concerns the converse problem, i.e., the inference of contact and environment propert ..."
Abstract

Cited by 20 (3 self)
 Add to MetaCart
(Show Context)
Motion and interaction with the environment are fundamentally intertwined. Few peopletracking algorithms exploit such interactions, and those that do assume that surface geometry and dynamics are given. This paper concerns the converse problem, i.e., the inference of contact and environment properties from motion. For 3D human motion, with a 12segment articulated body model, we show how one can estimate the forces acting on the body in terms of internal forces (joint torques), gravity, and the parameters of a contact model (e.g., the geometry and dynamics of a springbased model). This is tested on motion capture data and videobased tracking data, with walking, jogging, cartwheels, and jumping. 1.
Tracking People Interacting with Objects
"... While the problem of tracking 3D human motion has been widely studied, most approaches have assumed that the person is isolated and not interacting with the environment. Environmental constraints, however, can greatly constrain and simplify the tracking problem. The most studied constraints involve ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
(Show Context)
While the problem of tracking 3D human motion has been widely studied, most approaches have assumed that the person is isolated and not interacting with the environment. Environmental constraints, however, can greatly constrain and simplify the tracking problem. The most studied constraints involve gravity and contact with the ground plane. We go further to consider interaction with objects in the environment. In many cases, tracking rigid environmental objects is simpler than tracking highdimensional human motion. When a human is in contact with objects in the world, their poses constrain the pose of body, essentially removing degrees of freedom. Thus what would appear to be a harder problem, combining object and human tracking, is actually simpler. We use a standard formulation of the body tracking problem but add an explicit model of contact with objects. We find that constraints from the world make it possible to track complex articulated human motion in 3D from a monocular camera. 1.
Videobased 3D Motion Capture through Biped Control
"... Figure 1: Controller Reconstruction from Video: We estimate biped controllers from monocular video sequences (top) together with a physicsbased responsive character (bottom left) that we can simulate in new environments (bottom right). Markerless motion capture is a challenging problem, particular ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
Figure 1: Controller Reconstruction from Video: We estimate biped controllers from monocular video sequences (top) together with a physicsbased responsive character (bottom left) that we can simulate in new environments (bottom right). Markerless motion capture is a challenging problem, particularly when only monocular video is available. We estimate human motion from monocular video by recovering threedimensional controllers capable of implicitly simulating the observed human behavior and replaying this behavior in other environments and under physical perturbations. Our approach employs a statespace biped controller with a balance feedback mechanism that encodes control as a sequence of simple control tasks. Transitions among these tasks are triggered on time and on proprioceptive events (e.g., contact). Inference takes the form of optimal control where we optimize a highdimensional vector of control parameters and the structure of the controller based on an objective function that compares the resulting simulated motion with input observations. We illustrate our approach by automatically estimating controllers for a variety of motions directly from monocular video. We show that the estimation of controller structure through incremental optimization and refinement leads to controllers that are more stable and that better approximate the reference motion. We demonstrate our approach by capturing sequences of walking, jumping, and gymnastics.
3D Reconstruction of a Smooth Articulated Trajectory from a Monocular Image Sequence ∗
"... An articulated trajectory is defined as a trajectory that remains at a fixed distance with respect to a parent trajectory. In this paper, we present a method to reconstruct an articulated trajectory in three dimensions given the two dimensional projection of the articulated trajectory, the 3D parent ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
An articulated trajectory is defined as a trajectory that remains at a fixed distance with respect to a parent trajectory. In this paper, we present a method to reconstruct an articulated trajectory in three dimensions given the two dimensional projection of the articulated trajectory, the 3D parent trajectory, and the camera pose at each time instant. This is a core challenge in reconstructing the 3D motion of articulated structures such as the human body because endpoints of each limb form articulated trajectories. We simultaneously apply activityindependent spatial and temporal constraints, in the form of fixed 3D distance to the parent trajectory and smooth 3D motion. There exist two solutions that satisfy each instantaneous 2D projection and articulation constraint (a ray intersects a sphere at up to two locations) and we show that resolving this ambiguity by enforcing smoothness is equivalent to solving a binary quadratic programming problem. A geometric analysis of the reconstruction of articulated trajectories is also presented and a measure of the reconstructibility of an articulated trajectory is proposed. 1.
Videobased hand manipulation capture through composite motion control
 ACM Transactions on Graphics (SIGGRAPH
, 2013
"... Copyright Notice ..."
3D Pose Tracking of Walker Users ’ Lower Limb with a StructuredLight Camera on a Moving Platform
"... Tracking and understanding human gait is an important step towards improving elderly mobility and safety. Our research team is developing a visionbased tracking system that estimates the 3D pose of a wheeled walker user’s lower limbs with a depth sensor, Kinect, mounted on the moving walker. Our tr ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Tracking and understanding human gait is an important step towards improving elderly mobility and safety. Our research team is developing a visionbased tracking system that estimates the 3D pose of a wheeled walker user’s lower limbs with a depth sensor, Kinect, mounted on the moving walker. Our tracker estimates 3D poses from depth images of the lower limbs in the coronal plane in a dynamic, uncontrolled environment. We employ a probabilistic approach based on particle filtering, with a measurement model that works directly in the 3D space and another measurement model that works in the projected image space. Empirical results show that combining both measurements, assuming independence between them, yields tracking results that are better than with either one alone. Experiments are conducted to evaluate the performance of the tracking system with different users. We demonstrate that the tracker is robust against unfavorable conditions such as partial occlusion, missing observations, and deformable tracking target. Also, our tracker does not require user intervention or manual initialization commonly required in most trackers. 1.