Are the object manipulation techniques traditionally used in head–mounted displays (HMDs) applicable to augmented reality based projection systems? This paper examines the differences between HMD – and projector/camera– based AR interfaces in the light of a manipulation task involving documents and applications projected on common office surfaces such as tables, walls, cabinets, and floor. We report a Wizard of Oz study where subjects were first asked to create gesture/voice commands to move 2D objects on those surfaces and then exposed to gestures created by the authors. Among the options, subjects could select the object to be manipulated using voice command; touching, pointing, and grabbing gesture; or a virtual mouse. The results show a strong preference for a manipulation interface based on pointing gestures using small hand movements and involving minimal body movement. Direct touching of the object was also common when the object being manipulated was within the subjects ’ arm reach. Based on these results, we expect that the preferred interface resembles, in many ways, the egocentric model traditionally used in AR.

Selection is a common task in Virtual Environment (VE) interfaces. Several techniques have been created to perform the task and two testbed studies have performed comparison studies between the techniques. In this study focusing on Raycasting selection, we created an understanding of the known factors influencing selection and create a model of selection time and angular error. We found evidence that users try to maximize visual feedback by purposefully creating angular error and developed an argument for a possible visual feedback parameter to explain behavior across multiple studies. 1.

Abstract—This research investigates the design of a low-cost 3D spatial interaction approach using the Wii Remote for immersive Head-Mounted Display (HMD) virtual reality. Current virtual reality applications that incorporate the Wii Remote are either desktop virtual reality applications or systems that use large screen displays. However, the requirements for an HMD virtual reality system differ from such systems. This is mainly because in HMD virtual reality, the display screen does not remain at a fixed location. The user views the virtual environment through display screens that are in front of the user’s eyes and when the user moves his/her head, these screens move as well. This means that the display has to be updated in realtime based on where the user is currently looking. Normal usage of the Wii Remote requires the controller to be pointed in a certain direction, typically towards the display. This is too restrictive for HMD virtual reality systems that ideally require the user to be able to turn around in the virtual environment. Previous work proposed a design to achieve this, however it suffered from a number of drawbacks. The aim of this study is to look into a suitable method of using the Wii Remote for 3D interaction in a space around the user for HMD virtual reality. This paper presents an overview of issues that had to be considered, the system design as well as experimental results. Keywords—3D interaction, head-mounted display, virtual reality, Wii remote. V

Abstract — There are a few fundamental pointing-based user interface techniques for performing selections in 3D environments. One of these techniques is ray pointing, which makes selections by determining intersections of objects along a single ray cast from the user. This technique is susceptible to inaccuracy of the user interface tracking technology and user noise (e.g., jittery hand, see actual viewing ray in Figure 1 (bottom) and how it misses the object when error is factored in). Another technique is the frustum-based (or equivalent cone-based) approach which casts a frustum from the user into the 3D scene. Selections are made by performing intersections of objects with the frustum. This technique resolves the imprecision associated with the ray pointing technique (see how the lightly shaded cone in Figure 1 (bottom) still intersects the object), but may produce multiple ambiguous selections as shown in Figure 2. This paper presents probabilistic selection algorithms and integration schemes that address some of the ambiguities associated with the frustum-based selection technique. The selection algorithms assign probabilities that the user has selected particular objects using a set of low-level 3D intersection-based selection techniques and the relationship of the objects in a hierarchical database. Final selections are generated by integating the outputs from each selection algorithm using one of several weighting schemes. We implemented the selection algoithms and weighting schemes within our distributed augmented reality (AR) and virtual reality (VR) architecture. Next, we performed several experiments to empirically evaluate the probabilistic selection techniques and integration schemes. Our results show that the combined selection and integration algorithms are effective at disambiguating multiple selections. This paper thoroughly describes the architecture, experiments, and results.

Complex 3D interaction in virtual environments may inhibit user interaction and cause frustration. Supporting adaptivity based on the detected user frustration can be considered as one promising solution to enhance user interaction. Our work proposes to provide adaptive assistance to users who are frustrated during their interaction with 3D user interfaces in virtual environments. The obtrusiveness of physiological measurements to detect frustration inspired us to investigate the pressure patterns exerted on a 3D input device for this purpose. The experiment presented in this paper has shown a great potential on utilizing the finger pressure measures as an alternative to physiological measures to indicate user frustration during interaction. Furthermore, the findings in this particular context showed that adaptation of haptic interaction was effective in increasing the user’s performance and making users feel less frustrated in performing their tasks in the 3D environment.