Introduction The field of Augmented Reality (AR) has existed for just over one decade, but the growth and progress in the past few years has been remarkable. In 1997, the first author published a survey [3] (based on a 1995 SIGGRAPH course lecture) that defined the field, described many problems, and summarized the developments up to that point. Since then, the field has grown rapidly. In the late 1990s, several conferences specializing in this area were started, including the International Workshop and Symposium on Augmented Reality [29], the International Symposium on Mixed Reality [30], and the Designing Augmented Reality Environments workshop. Some wellfunded interdisciplinary consortia were formed that focused on AR, notably the Mixed Reality Systems Laboratory [50] in Japan and Project ARVIKA [61] in Germany. A freely-available software toolkit (the ARToolkit) for rapidly building AR applications is now available [2]. Because of this wealth of new developments, an updated survey

Tracking is a very important research subject in a real-time augmented reality context. The main requirements for trackers are high accuracy and little latency at a reasonable cost. In order to address these issues, a real-time, robust, and efficient 3D modelbased tracking algorithm is proposed for a “video see through ” monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, nonlinear pose estimation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curves interaction matrices are given for different 3D geometrical primitives including straight lines, circles, cylinders, and spheres. A local moving edges tracker is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively reweighted least squares implementation. This approach is then extended to address the 3D model-free augmented reality problem. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination, and mistracking.

This paper presents a framework to achieve real-time augmented reality applications. We propose a framework based on the visual servoing approach well known in robotics. We consider pose or viewpoint computation as a similar problem to visual servoing. It allows one to take advantage of all the research that has been carried out in this domain in the past. The proposed method features simplicity, accuracy, efficiency, and scalability wrt. to the camera model as well as wrt. the features extracted from the image. We illustrate the efficiency of our approach on augmented reality applications with various real image sequences.

Tele-immersion is a new medium that enables a user to share a virtual space with remote participants. The user is immersed in a rendered 3D-world that is transmitted from a remote site. To acquire this 3D description we apply biand trinocular stereo techniques. The challenge is to compute dense stereo range data at high frame rates, since participants cannot easily communicate if the processing cycle or network latencies are long. Moreover, new views of the received 3D-world must be as accurate as possible. We address both issues of speed and accuracy and we propose a method for combining motion and stereo in order to increase speed and robustness.

We propose an efficient online real-time solution for singlecamera 3–D tracking of rigid objects that can handle large camera displacements, drastic aspect changes, and partial occlusions. While the offline camera registration problem can be considered as essentially solved, robust online tracking remains an open issue because many real-time algorithms described in the literature still lack robustness and are prone to drift and jitter. To solve these problems, we have developed a robust approach to 3–D feature matching that can handle widebaseline matching: our method merges the information from preceding frames in traditional recursive tracking fashion with that provided by a very limited number of keyframes created during an offline stage. This combination results in a system that does not suffer from the above difficulties and can deal with drastic aspect changes. We use Augmented Reality applications to demonstrate its behavior because they are particularly demanding in terms of tracking performance. 1.

Abstract. Many applications of 3D object recognition, such as augmented reality or robotic manipulation, require an accurate solution for the 3D pose of the recognized objects. This is best accomplished by building a metrically accurate 3D model of the object and all its feature locations, and then fitting this model to features detected in new images. In this chapter, we describe a system for constructing 3D metric models from multiple images taken with an uncalibrated handheld camera, recognizing these models in new images, and precisely solving for object pose. This is demonstrated in an augmented reality application where objects must be recognized, tracked, and superimposed on new images taken from arbitrary viewpoints without perceptible jitter. This approach not only provides for accurate pose, but also allows for integration of features from multiple training images into a single model that provides for more reliable recognition 1. 1

To achieve accurate registration, the transformations which locate the tracking system components with respect to the environment must be known. These transformations relate the base of the tracking system to the virtual world and the tracking system's sensor to the graphics display. In this paper we present a unified, general calibration method for calculating these transformations. A user is asked to align the display with objects in the real world. Using this method, the sensor to display and tracker base to world transformations can be determined with as few as three measurements.

Abstract. Location-based services like reminders, electronic graffiti, and tourist guides normally require a custom, location-sensitive database that must be custom-tailored for the application at hand. This deployment cost reduces the initial appeal of such services. However, there is much location-tagged data already available on the Web which can be easily used to create compelling location-aware applications with almost no deployment cost. Such tagged data can be used directly in applications as well as to provide evidence in models of activity. We describe three applications that take advantage of existing Web data combined with location measurements from a GPS receiver. The first application, “Pinpoint Search”, finds web pages of nearby places based on GPS coordinates, queries from a Web mapping service, and general Web searches. The second application, “XRay”, uses the mapping service to find businesses in a building by pointing a GPS-equipped electronic compass at the building. The third application is called “Travelogue”, and it builds a map and clickable points of interest to help automatically annotate a trip based on GPS coordinates. Finally, we discuss the use of Web-based data as rich sources of evidence for probabilistic models of a user’s activity, including a means for interpreting the explanation for the loss of Web signals as users enter structures. 1

Augmented Reality(AR) aims at merging the real and the virtual in order to enrich a real environment with virtual information. Augmentations range from simple text annotations accompanying real objects to virtual mimics of reallife objects inserted into a real environment. In the latter case the ultimate goal is to make it impossible to differentiate between real and virtual objects. Several problems need to be overcome before realizing this goal. Amongst them are the rigid registration of virtual objects into the real environment, the problem of mutual occlusion of real and virtual objects and the extraction of the illumination distribution of the real environment in order to render the virtual objects with this illumination model. This paper will unfold how we proceeded to implement an Augmented Reality System that registers virtual objects into a totally uncalibrated video sequence of a real environment that may contain some moving parts. The other problems of occlusion and illumination will not be discussed in this paper but are left as future research topics.

The successful development of autonomous robotic systems requires careful fusion of complex subsystems for perception, planning, and control. Often these subsystems are designed in a modular fashion and tested individually. However, when ultimately combined with other components to form a complete system, unexpected interactions between subsystems can occur that make it difficult to isolate the source of problems. This paper presents a novel paradigm for robot experimentation that enables unified testing of individual subsystems while acting as part of a complete whole made up of both virtual and real components. We exploit the recent advances in speed and accuracy of optical motion capture to localize the robot, track environment objects, and extract extrinsic parameters for moving cameras in real-time. We construct a world model representation that serves as ground truth for both visual and tactile sensors in the environment. From this data, we build spatial and temporal correspondences between virtual elements, such as motion plans, and real artifacts in the scene. The system enables safe, decoupled testing of component algorithms for vision, motion planning and control that would normally have to be tested simultaneously on actual hardware. We show results of successful online applications in the development of an autonomous humanoid robot. 1