In this paper, we introduce and analyze four gesture-controlled musical instruments. We briefly discuss the test platform designed to allow for rapid experimentation of new interfaces and control mappings. We describe our design experiences and discuss the effects of system features such as latency, resolution and lack of tactile feedback. The instruments use virtual reality hardware and computer vision for user input, and three-dimensional stereo vision as well as simple desktop displays for providing visual feedback. The instrument sounds are synthesized in real-time using physical sound modeling.

While many new interfaces for musical expression have been presented in the past, methods to evaluate these interfaces are rare. This paper presents a method and a study comparing the potential for musical expression of different string-instrument based musical interfaces. Cues for musical expression are defined based on results of research in musical expression and on methods for musical education in instrumental pedagogy. Interfaces are evaluated according to how well they are estimated to allow players making use of their existing technique for the creation of expressive music.

When learning a classical instrument, people often either take lessons in which an existing body of "technique" is delivered, evolved over generations of performers, or in some cases people will "teach themselves" by watching people play and listening to existing recordings. What does one do with a complex new digital instrument? In this paper I address this question drawing on my experience in learning several very di#erent types of sophisticated instruments: the Glove Talk II real-time gesture-to-speech interface, the Digital Marionette controller for virtual 3D puppets, and pianos and keyboards. As the primary user of the first two systems, I have spent hundreds of hours with Digital Marionette and Glove-Talk II, and thousands of hours with pianos and keyboards (I continue to work as a professional musician). I will identify some of the underlying principles and approaches that I have observed during my learning and playing experience common to these instruments. While typical accounts of users learning new interfaces generally focus on reporting beginner's experiences, for various practical reasons, this is fundamentally di#erent by focusing on the expert's learning experience.

www.robotcowboy.com “Why, he’s a reg’lar musicker! ” said Button-Bright. “What’s a musicker? ” asked Dorothy. “Him! ” said the boy. Hearing this, the fat man sat up a little stiffer than before, as if he had received a compliment, and still came the sounds: Tiddle-widdle-iddle, oom pom-pom, Oom pom-pom, oom— “Stop it! ” cried the shaggy man, earnestly. “Stop that dreadful noise.” The fat man looked at him sadly and began his reply. When he spoke the music changed and the words seemed to accompany the notes. He said — or rather sang: It isn’t a noise that you hear, But Music, harmonic and clear. My breath makes me play

Gestural controllers in the musical arts provide a unique perspective on human-computer interaction. Given the novelty of the field, there is little research into the long term usage of gestural controllers. In the course of comparing the Buchla Lightning and the Mathews Radio Baton, the authors questioned users of these interfaces with over 10 years experience. The answers can help performers understand the implications of using gestural controllers, as well as designers who are interested in the long term usage patterns of their interfaces. 1.