This paper borrows ideas from social science to inform the design of novel "sensing" user-interfaces for computing technology. Specifically, we present five design challenges inspired by analysis of human-human communication that are mundanely addressed by traditional graphical user interface designs (GUIs). Although classic GUI conventions allow us to finesse these questions, recent research into innovative interaction techniques such as `Ubiquitous Computing' and `Tangible Interfaces' has begun to expose the interaction challenges and problems they pose. By making them explicit we open a discourse on how an approach similar to that used by social scientists in studying human-human interaction might inform the design of novel interaction mechanisms that can be used to handle human-computer communication accomplishments.

Inertial measurement components, which sense either acceleration or angular rate, are being embedded into common user interface devices more frequently as their cost continues to drop dramatically. These devices hold a number of advantages over other sensing technologies: they measure relevant parameters for human interfaces and can easily be embedded into wireless, mobile platforms. The work in this dissertation demonstrates that inertial measurement can be used to acquire rich data about human gestures, that we can derive efficient algorithms for using this data in gesture recognition, and that the concept of a parameterized atomic gesture recognition has merit. Further we show that a framework combining these three levels of description can be easily used by designers to create robust applications.

We report research performed on gesture analysis and mapping to music. Various movements were recorded using 3D optical motion capture. Using this system, we produced animations from movements/dance, and generate in parallel the soundtrack from the dancer's movements. Prior to the actual sound mapping process, we performed various motion analyses. We present here two methods, both independent of specific orientation or location of the subject. The first deals with gestural segmentation, while the second uses pattern recognition.

Abstract not found

Abstract not found

A multi-user, polyphonic sensor stage environment that maps position and gestures of up to four performers to the pitch and articulation of distinct notes is presented. The design seeks to provide multiple players on a stage with the feeling of a traditional acoustic instrument by giving them complete control over the instrument's expressive parameters and a clear causal connection between their actions and the resulting sound. The positions of the performers are determined by a custom ultrasonic tracking system, while hand motions are measured by custom-made gloves containing accelerometer units. Furthermore, juggling clubs are illuminated dynamically to make complex juggling patterns more apparent. The system is currently on tour with the Flying Karamazov Brothers juggling troupe. # Present address: ThingMagic LLC, Cambridge, MA 02142 # Present address: Trimble Navigation Ltd, Sunnyvale, CA 94088 CR Categories: B.4.2 [Input/output and data communications]: input/output devices---Channels and controllers; J.5 [Arts and Humanities ]: Performing Arts Keywords: Applications, HCI (Human-Computer Interface), Object Tracking, Spatialized Sound, User Interface Design 1

This article describes experiments conducted with the system and 3- to 5-year-old children. We highlight several dimensions of the study pertaining to music education, including attention span, spontaneous development of playing modes, and capacity to listen analytically. We describe very encouraging preliminary results and stress the importance of using reflective interactive systems for triggering musical interest in children and creating stimulating, nonsupervised music learning environments. We conclude by setting up our research in the context of the theory of flow as an optimal experience.

Recent advances in computer video projection open up new possibilities for real-time interactive, persuasive displays. Now a display can continuously adapt to a viewer so as to maximize its effectiveness. However, by the very nature of persuasion, these displays must be both immersive and subtle. We have been working on technologies that support this application including multiprojector and implicit interaction techniques. These technologies have been used to create a series of interactive persuasive displays that are described.

Drum controllers designed by researchers and commercial companies use a variety of techniques for capturing percussive gestures. It is challenging to obtain both quick response times and low-level data (such as position) that contain expressive information. This research is a comprehensive study of current methods to evaluate the available strategies and technologies. This study aims to demonstrate the benefits and detriments of the current state of percussion controllers as well as yield tools for those who would wish to conduct this type of study in the future.

We present a multimedia project incorporating music and dance. We used a 3D motion capture system to produce animations from dance and generate the soundtrack from the dancer's movements. Movement analysis is performed to extract the important features of a particular gesture. Based on the parameters chosen from this analysis, various mappings between gesture and music are applied. In particular, the motion capture data is used to trigger and modify the timbre of sounds. This paper describes the method and the interactive environment that are under development.