A growing body of research shows several advantages to multimodal interfaces including increased expressiveness, flexibility, and user freedom. This paper investigates the design of such an interface that integrates speech and hand gestures. The interface has the additional property of operating relative to the user and can be used while the user is in motion or standing at a distance from the computer display. The paper then describes an implementation of the multimodal interface for a whole Earth 3D visualization which presents navigation interface challenges due to the large magnitude of scale and extended spaces that are available. The characteristics of the multimodal interface are examined, such as speed, recognizability of gestures, ease and accuracy of use, and learnability under likely conditions of use. This implementation shows that such a multimodal interface can be effective in a real environment and sets some parameters for the design and use of such interfaces. 1.

We introduce the term „Anywhere Augmentation ” to refer to the idea of linking location-specific computing services with the physical world, making them readily and directly available in any situation and location. This chapter presents a novel approach to „Anywhere Augmentation“ based on efficient human input for wearable computing and augmented reality (AR). Current mobile and wearable computing technologies, as found in many industrial and governmental service applications, do not routinely integrate the services they provide with the physical world. Major limitations in the computer’s general scene understanding abilities and the infeasibility of instrumenting the whole globe with a unified sensing and computing environment prevent progress in this area. Alternative approaches must be considered. We present a mobile augmented reality system for outdoor annotation of the real world. To reduce user burden, we use openly available aerial photographs in addition to the wearable system’s usual data sources (position, orientation, camera and user input). This allows the user to accurately annotate 3D features from a single position by aligning features in both their firstperson viewpoint and in the aerial view. At the same time, aerial photographs provide a rich set of features that can be automatically extracted to create best guesses of intended annotations with minimal user input. Thus, user interaction is often as simple as casting a ray from a firstpersonview, and then confirming the feature from the aerial view. We examine three types of aerial photograph features – corners, edges, and regions – that are suitable for a wide variety of useful mobile augmented reality applications. By using aerial photographs in combination with wearable augmented reality, we are able to achieve much higher accuracy 3D annotation positions from a single user location than was previously possible. 1

We have designed a mobile-PDA-based- interface for real-time control of virtual characters in multiuser semi-immersive Virtual Environments-using a large rear-projection screen. The proof-of-concept implementation we present shows the potential of handheld devices as powerful interfaces to Virtual Reality applications. This technique eliminates the display of floating menus and other widgets over the simulation screen. A brief discussion on the advantages and disadvantages of using a handheld for 3D interaction is presented as well.

This paper presents and evaluates a set of pictorial depth cues for far-field outdoor mobile augmented reality (AR). We examine the problem of accurately placing virtual annotations at physical target points from a static point of view. While it is easy to line up annotations with a target point's projection in the view plane, finding the correct distance for the annotation is difficult if the target point is not represented in an environment model. We have found that AR depth cues, such as vertical and horizontal shadow planes, a small top-down map, or color encodings of relative depth, have a positive impact on a user's ability to align a 3D cursor with physical objects at various distances. These cues aid the user's depth perception and estimation by providing information about the 3D cursor's distance and its relationship in 3-space to any features that may already have been annotated. We conducted a user study that measures the effects of different depth cues for both absolute 3D cursor placement as well as placement relative to a small number of marked reference points, whose distances are known. Our study provides insight about mobile AR users ' ability to judge distances both absolutely and relatively, and we identify techniques that successfully enhance their performance. 1.

We present a mobile augmented reality system for outdoor annotation of the real world. To reduce user burden, we use aerial photographs in addition to the wearable system’s usual data sources (position, orientation, camera and user input). This allows the user to accurately annotate 3D features with only a few simple interactions from a single position by aligning features in both their firstperson viewpoint and in the aerial view. We examine three types of aerial photograph features – corners, edges, and regions – that are suitable for a wide variety of useful mobile augmented reality applications, and are easily visible on aerial photographs. By using aerial photographs in combination with wearable augmented reality, we are able to achieve much higher accuracy 3D annotation positions than was previously possible from a single user location.

This paper presents techniques designed to facilitate interaction at a distance in mobile augmented reality and evaluates different input controls for them. The goal of the techniques is to quickly and accurately annotate distant physical objects not yet represented in the computer’s model of the scene. To this end, we present a new method for judging virtual object distance. Using virtual depth cues, the user can accurately judge depth values without having to resort to near-field-only cues such as stereo vision and parallax. To place new annotations, we need to position a virtual 3D cursor accurately relative to known landmarks. We developed four techniques for controlling this cursor and conducted a user study on the relative efficiency of these techniques.

Figure 1: An example color panorama and semi-automatically generated depth map pair. Darker regions of the depth map are closer to the user. To generate both images the user simply has to look around the scene. Both images are composed of the four surrounding faces of a cube map, and are not warped to a cylindrical projection. This cube projection causes the strange peak on the roof line in the center of the images. This paper presents methodology for integrating a small, singlepoint laser range finder into a wearable augmented reality system. We first present a way of creating object-aligned annotations with very little user effort. Second, we describe techniques to segment and pop-up foreground objects. Finally, we introduce a method using the laser range finder to incrementally build 3D panoramas from a fixed observer’s location. To build a 3D panorama semiautomatically, we track the system’s orientation and use the sparse range data acquired as the user looks around in conjunction with real-time image processing to construct geometry around the user’s position. Using full 3D panoramic geometry, it is possible for new

This paper describes a novel 3D interaction technique called the “SkeweR”, dedicated to the 2-user collaborative manipulation of objects in virtual environments. This technique enables two users to move simultaneously the same virtual object in 3D. For this aim, each user manipulates the object by one crushing point, like handling the extremity of a skewer. The SkeweR uses only translation information from the users ’ motions to change both the position and orientation of the manipulated object. By using more crushing points, this technique could easily be extended to 3 or more users. Thus, the SkeweR technique could be used to improve the collaborative manipulation of objects in numerous applications of Virtual Reality, such as: virtual prototyping, maintenance and training simulations, architectural mock-up reviews, etc.

Human judgment is an integral part of the teleoperation process that is often heavily influenced by a single video feed returned from the remote environment. Poor camera placement, narrow field of view, and other camera properties can significantly impair the operator's perceptual link to the environment, inviting cognitive mistakes and general disorientation. These faults may be enhanced or muted, depending on the camera mountings and control opportunities that are at the disposal of the operator. These issues form the basis for two user studies that assess the effectiveness of existing and potential teleoperation controls. Findings suggest that providing a camera that is controlled independently from the orientation of the vehicle may yield significant benefits. Moreover, there is evidence to support the use of separate cameras for different navigational subtasks. Third, the use of multiple cameras can also be used to provide assistance without encroaching on the operator's desired threshold for control.

This paper evaluates different display devices for selection or annotation tasks in augmented reality (AR). We compare three different display types – a head mounted display and two hand held displays. The first hand held display is configured as a magic lens where the user sees the augmented space directly behind the display. The second hand held display is configured to be used at waist level (as one would commonly hold a tablet computer) but the view is still of the scene in front of the user. Making a selection or annotation in AR requires two distinct tasks by the user. First, the user must find the real (or virtual) object they want to mark. Second, the user must move a cursor to the object’s location. We test and compare our three representative displays with respect to both tasks. We found that using a hand held display in the magic lens configuration was faster for cursor movement than either of the other two displays. There was no significant difference among the displays regarding the amount of time it took users to search for either physical or virtual objects.