We present a Dynamic Spatially Augmented Reality system for augmenting movable 3D objects in an indoor environment using multiple projectors. We describe a real-time system for applying virtual paint and textures to real objects simply by direct physical manipulation of the object and a "paint brush" stylus. We track the objects and the "paintbrush", and illuminate the objects with images that remain registered as they move, to create the illusion of material properties. The system is simple to use and we hope it may herald new applications in diverse fields such as visualization, tele-immersion, art and architecture. The system currently works with tracked objects whose geometry was pre-acquired and models created manually, but it is possible to extend it, by adding cameras to the environment, to acquire object geometry automatically and use vision-based tracking for the object and paintbrush.

Abstract: The usability of three-dimensional (3D) interaction techniques depends upon both the interface software and the physical devices used. However, little research has addressed the issue of mapping 3D input devices to interaction techniques and applications. This is especially crucial in the field of Virtual Environments (VEs), where there exists a wide range of potential 3D input devices. In this paper, we discuss the use of Pinch Gloves Y – gloves that report contact between two or more fingers – as input devices for VE systems. We begin with an analysis of the advantages and disadvantages of the gloves as a 3D input device. Next, we present a broad overview of three novel interaction techniques we have developed using the gloves, including a menu system, a text input technique, and a two-handed navigation technique. All three of these techniques have been evaluated for both usability and task performance. Finally, we speculate on further uses for the gloves.

Mediated reality describes the concept of filtering our vision of reality, typically using a head-worn video mixing display. In this paper, we propose a generalized concept and new tools for interactively mediated reality. We present also our first prototype system for painting, grabbing and glueing together real and virtual elements. 1.

This article provides a snapshot of immersive virtual reality(IVR) use for scientific visualization, in the context of the evolution of computing in general and of user interfaces in particular. The main thesis of this article is that IVR has great potential for dealing with the serious problem of exponentiallygrowing scientific datasets. Our abilityto produce large datasets both through numerical simulation and through data acquisition via sensors is outrunning our abilityto make sense of those datasets. While our idea of "large" datasets used to be measured in hundreds of gigabytes, based at least in part on what we could easilystore, manipulate, and displayin real time, today's science and engineering are producing terabytes and soon even petabytes, both from observation via sensors and as output from numerical simulation. Clearly, visualization by itself will not solve the problem of understanding truly large datasets that would overwhelm both displaycapacityand the human visual system. We advocate a human--computer partnership that draws on the strengths of each partner, with algorithmic culling and feature-detection used to identifythe small fraction of the data that should be visuallyexamined in detail bythe human. Our hope is that IVR will be a potent tool to let humans "see" patterns, trends, and anomalies in their data well beyond what they can do with conventional 3D desktop displays. r 2002 Published b Elsevier Science Ltd.

We present an interactive approach to non-photorealistic rendering that contrasts with the standard black box character of previous rendering techniques in that observation and interaction take place during rendering. Our technique is based on the idea of approaching non-photorealistic rendering by modeling with rendering primitives. This new approach supports interruption, tweaking, manipulation, and re-direction of the rendering as it develops. While we draw upon computational support for primitive placement to avoid having to painstakingly place each pixel, we limit the computational influence to enable freedom of interaction with the elements. We implement this new paradigm in a stroke-based rendering application using a stack of interaction buffers to store attributes of the primitives during the rendering. By manipulating the data in these buffers we affect the behavior of strokes on the canvas. This allows us to create and adjust images in non-photorealistic styles such as painterly rendering, pointillism, and decorative mosaics at interactive frame rates.

We present examples of four different custom built input devices and discuss their benefits to virtual environment interfaces. In each case, we describe the underlying motivation behind the device's creation and present interaction techniques that were made possible by each device's unique characteristics.

We present a novel class of virtual reality input devices that combine pop through buttons with 6 DOF trackers. Compared to similar devices that use conventional buttons, pop through devices double the number of potential discrete interaction modes, since each button has two activation states corresponding to light and firm pressure. This additional state per button provides a foundation to address a range of shortcomings with conventional virtual environment input devices that includes reducing the physical dexterity required to perform interactions, reducing the cognitive complexity of some compound tasks, and enabling the design of less obtrusive devices without sacrificing expressive power. Specifically, we present two novel input devices: the FingerSleeve was designed to be minimally obtrusive physically, whereas the TriggerGun was designed to be physically similar to, yet more functional than a conventional hand-held trigger device. Further, we present a set of novel navigation and interaction techniques that leverage the capabilities of our pop through button devices to improve interaction quality, and provide insight into harnessing the potential of pop through buttons for other tasks. Finally, we discuss how we incorporated one of our devices into a real application. 1

This paper proposes a user-friendly modeling system that interactively generates 3D organic-like shapes from user drawn sketches. A skeleton, in the form of a graph of branching polylines and polygons, is first extracted from the user's sketch. The 3D shape is then defined as a convolution surface generated by this skeleton. The skeleton's resolution is adapted according to the level of detail selected by the user. The subsequent 2D strokes are used to infer new object parts, which are combined with the existing shape using CSG operators. We propose an algorithm for computing a skeleton defined as a connected graph of polylines and polygons. To combine the primitives we propose precise CSG operators for a convolution surfaces blending hierarchy. Our new formulation has the advantage of requiring no optimization step for fitting the 3D shape to the 2D contours. This yields interactive performances and avoids any non-desired oscillation of the reconstructed surface. As our results show, our system allows nonexpert users to generate a wide variety of free form shapes with an easy to use sketch-based interface.

This paper presents a novel interaction paradigm to support musical performance using spatial audio. This method reduces the interface bottleneck between artistic intent and spatial sound rendering and allows dynamic positioning of sounds in space. The system supports collaborative performance, allowing multiple artists to simultaneously control the audio spatialization. The interface prototype is built upon standard virtual reality software and user interface technology. Tracked data gloves are used to manipulate audio objects and stereoscopic projection to display the virtual 3D sound stage.

Art in particular painting, has had clear impacts on the style, techniques, and processes of scientific visualization. Artists strive to create visual