Current user interface techniques such as WIMP or the desktop metaphor do not support real world tasks, because the focus of these user interfaces is only on human-computer interactions, not on human-real world interactions. In this paper, we propose a method of building computer augmented environments using a situation-aware portable device. This device, called NaviCam, has the ability to recognize the user's situation by detecting color-code IDs in real world environments. It displays situation sensitive information by superimposing messages on its video see-through screen. Combination of ID-awareness and portable video-see-through display solves several problems with current ubiquitous computers systems and augmented reality systems.

This paper addresses the problem of correcting visual registration errors in video-based augmented reality systems. Accurate visual registration between real and computergenerated objects in combined images is critically important for conveying the perception that both types of object occupy the same 3-dimensional (3D) space. Previous augmented reality systems concentrated on improving 3D coordinate system registration in order to reduce image registration error. This paper introduces the idea of dynamically measuring 2D registration error in combined images and using that information to correct 3D coordinate system registration error which in turn improves registration in the combined images. Registration can be made exact for one or more points in every combined image if a small video delay can be tolerated. Our experimental system achieves improved image registration, stability, and error tolerance from tracking system drift and jitter over current augmented reality systems. Compute...

This paper proposes a novel way to realize augmented reality (AR) systems. Unlike previous AR systems, which use head-mounted or head-up displays, our approach employs a palmtop-sized video see-through device to present computer augmented view of the real world. This configuration, which we call the magnifying glass approach, has several advantages over traditional head-up or head-mounted configurations. A user doesn't have to put on any cumbersome head gear. Like a magnifying glass, the user can easily move the device around and compare real and augmented images. We have built a prototype augmented reality system called NaviCam, based on the proposed magnifying glass approach. NaviCam uses a miniature gyro sensor to determine the orientation of device. It also has a vision-based ID recognition capability to detect a rough position of the device in the real world, and real world objects in front of the device. The experiment conducted by using NaviCam shows the great potential of the v...

: We introduce a local collaborative environment for gaming. In our setup multiple users can interact with the virtual game and the real surroundings at the same time. They are able to communicate with other players during the game. We describe an augmented reality setup for multiple users with see-trough head-mounted displays, allowing dedicated stereoscopic views and individualized interaction for each user. We use face-snapping for fast and precise direct object manipulation. With face snapping and the subdivision of the gaming space into spatial regions, the semantics of actions can be derived out of geometric actions of the user. Further, we introduce a layering concept allowing individual views onto the common data structure. The layer concept allows to make privacy management very easy by simply manipulating the common data structure. Moreover, assigning layers to spatial regions carefully, a special privacy management is often not necessary. Moving objects from one region into ...

We propose a practical application of wearable computing and augmented reality which enhances the game of billiards. A vision algorithm is implemented which operates in interactive-time with the user to assist planning and aiming. Probabilistic color models and symmetry operations are used to localize the table, pockets and balls through a video camera near the user's eye. Classification of the objects of interest is performed and each possible shot is ranked in order to determine its relative usefulness. The system allows the user to proceed through a regular pool game while it automatically determines strategic shots. The resulting trajectories are rendered as graphical overlays on a head mounted live video display. The wearable video output and the computer vision system provide an integration of real and virtual environments which enhances the experience of playing and learning the game of billiards without encumbering the player.

DyPERS, 'Dynamic Personal Enhanced Reality System', is a wearable system which uses augmented reality and computer vision to autonomously retrieve 'media memories' based on associations with real objects the user encounters. These are evoked as audio and video clips taken by the user and overlayed on top of real objects the user looks at. The user's visual and auditory scene is stored in real-time by the system (upon request) and is then associated (by user input) with a snap shot of a visual object. The object acts as a key which is detected by a real-time vision system when it is in view, triggering DyPERS to playback the appropriate audio-visual sequence. The vision system is a probabilistic algorithm which is capable of discriminating between hundreds of everyday objects under varying viewing conditions (lighting, pose, etc.). The record-and-associate paradigm of the system has many potential applications. Results on the use of the system in a museum tour scenario are described.

Introduction Computer technology has only recently become advanced enough to solve the problems it creates with its own interface. One solution, virtual reality (VR), immediately raises fundamental issues in both semantics and epistemology. Broadly, virtual reality is that aspect of reality which people construct from information, a reality which is potentially orthogonal to the reality of mass. Within computer science, VR refers to interaction with computer generated spatial environments, environments constructed to include and immerse those who enter them. VR affords non-symbolic experience within a symbolic environment. Since people evolve in a spatial environment, our knowledge skills are anchored to interactions within spatial environments. VR design techniques, such as scientific visualization, map digital information onto spatial concepts. When our senses are immersed in stimuli from the virtual world, our minds construct a closure to crea

DyPERS, 'Dynamic Personal Enhanced Reality System', uses augmented reality and computer vision to autonomously retrieve 'media memories' based on associations with real objects the user encounters. These are evoked as audio and video clips relevant for the user and overlayed on top of real objects the user encounters. The system utilizes an adaptive, audio-visual learning system on a tetherless wearable computer. The user's visual and auditory scene is stored in real-time by the system (upon request) and is then associated (by user input) with a snap shot of a visual object. The object acts as a key such that when the real-time vision system detects its presence in the scene again, DyPERS plays back the appropriate audio-visual sequence. The vision system is a probabilistic algorithm which is capable of discriminating between hundreds of everyday objects under varying viewing conditions (lighting, view changes, etc.). Once an audio-visual clip is stored, the vision system automatically...

Augmented Reality overlays computer generated images over the real world. To correctly depict spatial relations between real and virtual objects, these images have to be generated using transformations which correctly project a point in virtual space onto its corresponding point in the real world. This requires precise knowledge of the viewing projection of the head-mounted display (HMD) and its position. Methods to calibrate the projection parameters of the virtual cameras to that of the HMD have been presented in previous work. Most of these methods are complicated or use special equipment for the calibration process. We present a simple and fast calibration scheme, which does not require additional instrumentation or complicated procedures. This allows us to calibrate HMDs for specific users. The calibration process is presented to the user as interactively guided initialization step, which enables even inexperienced users to adapt the calibration to their eye distance and head ge...

Michael A. Bajura: Merging Real and Virtual Environments with Video See-Through Head-Mounted Displays (Under the direction of Henry Fuchs) This dissertation explores the problem of inserting virtual (computer-generated) objects into natural scenes in video-based augmented-reality (AR) systems. The augmented-reality system problems addressed are the shorter-term goal of making synthetic objects appear to be more stable and registered and the longer-term goal of presenting proper occlusion and interaction cues between real and synthetic objects. These results can be achieved in video-based AR systems by directly processing video images of the natural scene in real-time. No additional hardware or additional tracking mechanisms are needed above those currently used in augmented-reality systems. The implications of this work suggest a different approach to the design of AR tracking systems in which the user's world view is used as part of the tracking process. This work shows that trackin...