Results 1 - 10
of
34
Accurate Real-time Tracking Using Mutual Information
, 2010
"... In this paper we present a direct tracking approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of the displacement that makes it adapted to augmented reality applications. MI is a measure of the quantity ..."
Abstract
-
Cited by 21 (4 self)
- Add to MetaCart
In this paper we present a direct tracking approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of the displacement that makes it adapted to augmented reality applications. MI is a measure of the quantity of information shared by signals that has been widely used in medical applications. Since then, and although MI has the ability to perform robust alignment with illumination changes, multi-modality and partial occlusions, few works propose MI-based applications related to object tracking in image sequences due to some optimization problems. In this work, we propose an optimization method that is adapted to the MI cost function and gives a practical solution for augmented reality application. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the tracking results are far more robust and accurate than the existing solutions. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed. The ESM and the proposed MI tracking approaches are compared on a standard dataset. We also show the robustness of the proposed approach on registration applications with different sensor modalities: map versus satellite images and satellite images versus airborne infrared images within different AR applications.
The City of Sights: Design, Construction, and Measurement of an Augmented Reality Stage Set
"... views of the City of Sights, showing a virtual and a real representation of the total assembly. We describe the design and implementation of a physical and virtual model of an imaginary urban scene — the “City of Sights ” — that can serve as a backdrop or “stage ” for a variety of Augmented Reality ..."
Abstract
-
Cited by 14 (8 self)
- Add to MetaCart
(Show Context)
views of the City of Sights, showing a virtual and a real representation of the total assembly. We describe the design and implementation of a physical and virtual model of an imaginary urban scene — the “City of Sights ” — that can serve as a backdrop or “stage ” for a variety of Augmented Reality (AR) research. We argue that the AR research community would benefit from such a standard model dataset which can be used for evaluation of such AR topics as tracking systems, modeling, spatial AR, rendering tests, collaborative AR and user interface design. By openly sharing the digital blueprints and assembly instructions for our models, we allow the proposed set to be physically replicable by anyone and permit customization and experimental changes to the stage design which enable comprehensive exploration of algorithms and methods. Furthermore we provide an accompanying rich dataset consisting of video sequences under varying conditions with ground truth camera pose. We employed three different ground truth acquisition methods to support a broad range of use cases. The goal of our design is to enable and improve the replicability and evaluation of future augmented reality research. 1
Gravity-Aware Handheld Augmented Reality
- In Proc. IEEE/ACM ISMAR
, 2011
"... Figure 1: Gravity-awareness in handheld AR: detecting vertical surfaces, such as building façades, improves when using gravity-aligned feature descriptors (GAFD) [12] (left). For horizontal surfaces, such as a magazine on a table (center), we introduce gravity-rectified feature descriptors (GREFD) ..."
Abstract
-
Cited by 14 (3 self)
- Add to MetaCart
(Show Context)
Figure 1: Gravity-awareness in handheld AR: detecting vertical surfaces, such as building façades, improves when using gravity-aligned feature descriptors (GAFD) [12] (left). For horizontal surfaces, such as a magazine on a table (center), we introduce gravity-rectified feature descriptors (GREFD) that describe a feature based on a gravity-rectified camera image. We also show, how inertial sensors enable full 6 DoF pose estimation from horizontal surfaces with an occlusion-invariant edge-based detection method that only supports similarity transforms (right). This paper investigates how different stages in handheld Aug-mented Reality (AR) applications can benefit from knowing the di-rection of the gravity measured with inertial sensors. It presents ap-proaches to improve the description and matching of feature points, detection and tracking of planar templates, and the visual quality of the rendering of virtual 3D objects by incorporating the gravity vector. In handheld AR, both the camera and the display are located in the user’s hand and therefore can be freely moved. The pose of the camera is generally determined with respect to piecewise planar objects that have a known static orientation with respect to gravity. In the presence of (close to) vertical surfaces, we show how gravity-aligned feature descriptors (GAFD) improve the initializa-tion of tracking algorithms relying on feature point descriptor-based approaches in terms of quality and performance. For (close to) horizontal surfaces, we propose to use the gravity vector to rec-tify the camera image and detect and describe features in the rec-tified image. The resulting gravity-rectified feature descriptors (GREFD) provide an improved precision-recall characteristic and enable faster initialization, in particular under steep viewing angles. Gravity-rectified camera images also allow for real-time 6 DoF pose estimation using an edge-based object detection algorithm handling only 4 DoF similarity transforms. Finally, the rendering of virtual 3D objects can be made more realistic and plausible by taking into account the orientation of the gravitational force in addition to the relative pose between the handheld device and a real object. 1
Temporal Calibration in Multisensor Tracking Setups
"... Spatial tracking is one of the most challenging parts of Augmented Reality. Many AR applications rely on the fusion of several tracking systems in order to optimize the overall performance. While the topic of sensor fusion has already seen considerable interest, most results only deal with the integ ..."
Abstract
-
Cited by 6 (3 self)
- Add to MetaCart
Spatial tracking is one of the most challenging parts of Augmented Reality. Many AR applications rely on the fusion of several tracking systems in order to optimize the overall performance. While the topic of sensor fusion has already seen considerable interest, most results only deal with the integration of particular setups. A crucial part of sensor fusion is the temporal alignment of the sensor signals, as sensors in general are not synchronized. We present a general method to calibrate the temporal offset between different sensors by applying the normalized cross correlation method.
Second order optimization of mutual information for real-time image registration
- IEEE Trans. on Image Processing
, 2012
"... Abstract—In this paper we present a direct image registration approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of a set of 2D motion parameters. MI is a measure of the quantity of information shared by s ..."
Abstract
-
Cited by 5 (1 self)
- Add to MetaCart
(Show Context)
Abstract—In this paper we present a direct image registration approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of a set of 2D motion parameters. MI is a measure of the quantity of information shared by signals. Although it has the ability to perform robust alignment with illumination changes, multi-modality and partial occlusions, few works propose MI-based applications related to spatio-temporal image registration or object tracking in image sequences due to some optimization problems that we will explain. In this work, we propose a new optimization method that is adapted to the MI cost function and gives a practical solution for real time tracking. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the registration results are far more robust and accurate than the existing solutions while the computation is cheaper. A new approach is also proposed to speed up the computation of the derivatives and keep an equivalent optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed. Index Terms—Mutual information, registration, tracking, optimization. I.
Benchmarking Inertial Sensor-Aided Localization and Tracking Methods
"... benchmark datasets, that do not contain any inertial sensor measurements. At the example of metaio’s template tracking benchmarking set [18], we show how synthesizing inertial sensor measurements from the ground truth poses enables the evaluation of such methods at virtually no extra cost while prov ..."
Abstract
-
Cited by 4 (1 self)
- Add to MetaCart
(Show Context)
benchmark datasets, that do not contain any inertial sensor measurements. At the example of metaio’s template tracking benchmarking set [18], we show how synthesizing inertial sensor measurements from the ground truth poses enables the evaluation of such methods at virtually no extra cost while providing comparable results to using real inertial sensor data, which was validated using the setup in the right photo. This paper investigates means to benchmark methods for camera pose localization and tracking that in addition to a camera image make use of inertial sensor measurements. In particular the direction of the gravity has recently shown to provide useful information to aid vision-based approaches making them outperform visiononly methods. Obviously, it is desirable to benchmark the performance of such methods and to compare them with state-of-the-art approaches, but to the best of our knowledge, all publicly available benchmarking datasets unfortunately lack gravity information. We present different simple means to generate one’s own benchmarks for inertial sensor-aided localization and tracking methods and most considerably show how existing datasets, that do not have inertial sensor data, can be exploited. We demonstrate how to evaluate Gravity-Aligned Feature Descriptors (GAFD) and Gravity-Rectified Feature Descriptors (GREFD) on an existing benchmark dataset with ground truth poses. By synthesizing gravity measurements from these poses we achieve similar results to using real sensor measurements at significantly less effort. Most importantly, the proposed procedure enables the comparison with existing evaluation results on the same data. The paper concludes with a requirements analysis and suggestions for the design of future benchmarking datasets for localization and tracking methods. 1
Bidirectional composition on Lie groups for gradient-based image alignment
- IEEE Transactions on Image Processing
, 2010
"... Abstract—In this paper, a new formulation based on Bidi-rectional Composition on Lie Groups (BCL) for parametric gradient-based image alignment is presented. Contrary to the conventional approaches, the BCL method takes advantage of the gradients of both template and current image without combining ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
Abstract—In this paper, a new formulation based on Bidi-rectional Composition on Lie Groups (BCL) for parametric gradient-based image alignment is presented. Contrary to the conventional approaches, the BCL method takes advantage of the gradients of both template and current image without combining them a priori. Based on this bidirectional formulation, two methods are proposed and their relationship with state-of-the-art gradient based approaches is fully discussed. The first one, i.e. the BCL method, relies on the compositional framework to provide the minimization of the compensated error with respect to an augmented parameter vector. The second one, the Projected BCL (PBCL), corresponds to a close approximation of the BCL approach. A comparative study is carried out dealing with computational complexity, convergence rate and frequence of convergence. Numerical experiments using a conventional benchmark show the performance improvement especially for asymmetric levels of noise, which is also discussed from a theoretical point of view. Index Terms—Bidirectional image alignment, image registra-tion, gradient methods, Lie groups. I.
1 Realtime Registration-Based Tracking via Approximate Nearest Neighbour Search
"... Abstract—We introduce a new 2D visual tracking algorithm that utilizes an approximate nearest neighbour search to estimate per-frame state updates. We experimentally demonstrate that the new algorithm capable of estimating larger per-frame motions than the standard registration-based algorithms and ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
Abstract—We introduce a new 2D visual tracking algorithm that utilizes an approximate nearest neighbour search to estimate per-frame state updates. We experimentally demonstrate that the new algorithm capable of estimating larger per-frame motions than the standard registration-based algorithms and that it is more robust in a vision-controlled robotic alignment task. focusing on a few select regions. Often, this makes it more accurate than other methods. In this work we introduce and experimentally validate a new state estimation method for registration based tracking that utilizes a nearest neighbour search to update the warp parameters from one frame to the next. I.
Experiences with the Impact of Tracking Technology in Mobile Augmented Reality Evaluations
- In MobileHCI 2012 Workshop MobiVis, ACM
, 2012
"... In this paper, we discuss the impact of tracking technology on user studies of mobile augmented reality applications. We present findings from several of our previous publications in the field, discussing how tracking technology can impact, influence and compromise experimental results. Lessons lear ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
(Show Context)
In this paper, we discuss the impact of tracking technology on user studies of mobile augmented reality applications. We present findings from several of our previous publications in the field, discussing how tracking technology can impact, influence and compromise experimental results. Lessons learned from our experience show that suitable tracking technology is a key requirement and a fundamental factor in the user experience of the application. Tracking technology should therefore be considered not only during implementation but also as a factor in the design and evaluation phases. Author Keywords tracking, mobile augmented reality, evaluation
