The ability of a normal human listener to recognize objects in the environment from only the sounds they produce is extraordinarily robust with regard to characteristics of the acoustic environment and of other competing sound sources. In contrast, computer systems designed to recognize sound sources function precariously, breaking down whenever the target sound is degraded by reverberation, noise, or competing sounds. Robust listening requires extensive contextual knowledge, but the potential contribution of sound-source recognition to the process of auditory scene analysis has largely been neglected by researchers building computational models of the scene analysis process. This thesis proposes a theory of sound-source recognition, casting recognition as a process of gathering information to enable the listener to make inferences about

This paper reviews the existing literature on input device evaluation and design in human-computer interaction (HCI) and discusses possible applications of this knowledge to the design and evaluation of new interfaces for musical expression. Spression.5; a set of musical tasks is suggested to allow the evaluation of different existing controllers.

The Theremin was one of the first electronic musical instruments, yet it provides a degree of expressive real-time control that remains lacking in most modern electronic music interfaces. Underlying the deceptively simple capacitance measurement used by it and its descendants are a number of surprisingly interesting current transport mechanisms that can be used to inexpensively, unobtrusively, robustly, and remotely detect the position of people and objects. We review the relevant physics, describe appropriate measurement instrumentation, and discuss applications that began with capturing virtuosic performance gesture on traditional stringed instruments and evolved into the design of new musical interfaces. 1)

Abstract: Traditional musical instruments provide compelling metaphors for human-computer interfacing, both in terms of input (physical, gestural performance activities) and output (sound diffusion). The violin, one of the most refined and expressive of traditional instruments, combines a peculiar physical interface with a rich acoustic diffuser. We have built a new instrument that includes elements of both the violin's physical performance interface and its spatial filtering audio diffuser, yet eliminates both the resonating body and the strings. The instrument, BoSSA (Bowed-Sensor-Speaker-Array), is an amalgamation and extension of our previous work with violin interfaces, physical models, and directional tonal radiation studies. In addition to describing the various physical and software elements that make up BoSSA, we discuss some of its musical features and potentials; we are particularly impressed by the sense of presence and intimacy it provides, and by its potential for creating a new kind of electronic chamber music. Traditional musical instruments provide compelling metaphors for human-computer interfacing,

Abstract not found

This thesis presents a new computer interface metaphor for the real-time and simultaneous performance of dynamic imagery and sound. This metaphor is based on the idea of an inexhaustible, infinitely variable, time-based, audiovisual “substance ” which can be gesturally created, deposited, manipulated and deleted in a free-form, non-diagrammatic image space. The interface metaphor is exemplified by five interactive audiovisual synthesis systems whose visual and aural dimensions are deeply plastic, commensurately malleable, and tightly connected by perceptuallymotivated mappings. The principles, patterns and challenges which structured the design of these five software systems are extracted and discussed, after which the expressive capacities of the five systems are compared and evaluated.

The MPEG-4 standard, which will be published in October 1998, contains extensive provisions for sound synthesis as well as traditional methods of audio compression. At the heart of MPEG-4 Structured Audio, the sound-synthesis framework, is a new music-synthesis language called SAOL. This language, based on the Music-N model, is specified and defined fully in the MPEG-4 International Standard; a real-time implementation must be embedded in any device conforming to the full MPEG-4 Audio standard. In this paper, the structure and capabilities of SAOL are discussed, especially in comparison with other music languages. A discussion of the role of international standardization in the future development of computer-music tools is also presented.

This paper describes three different types of real-time optical tracking systems developed at the MIT Media Laboratory for use as expressive human-computer interfaces in music, dance, and interactive multimedia performances. Two of these, a multimodal conducing baton and a scanning laser rangefinder, are essentially hardware-based, while the third is a computer vision system that can identify and track different segments of the performers body. We discuss the technical concepts behind these devices and outline their applications in music and dance environments. 1.

This paper provides a review of gestural control of sound synthesis in the context of the design and evaluation of digital musical instruments. It discusses research in various areas related to this field and equally focuses on four main topics: analysis of music performers’ gestures, gestural capture technologies, real-time sound synthesis methods, and strategies for mapping gesture variables to sound synthesis input parameters. Finally, this approach is illustrated by presenting an application of this research to the control of digital audio effects.

A new breed of mobile phones has been designed to enable concurrent vibration and audio stimulation, or audiohaptics. This paper aims to share techniques for creating and optimizing audio-haptic effects to enhance the user interface. The authors present audio manipulation techniques specific to the multifunction transducer (MFT) technology. In particular two techniques, the Haptic Inheritance and Synthesis and Matching methods are discussed. These two methods of haptic media generation allow simple creation of vibration content, and also allow for compatibility with non-haptic mobile devices. The authors present preliminary results of an evaluation of 42 participants comparing audio-based haptic user interface (UI) feedback with audio-only feedback. The results show that users were receptive to audio-haptic UI feedback. The results also suggest that audio-haptics seems to enhance the perception of audio quality.