This paper will present observations on the design, artistic, and human factors of creating digital music controllers. Specific projects will be presented, and a set of design principles will be supported from those examples.

Recent research in physical modeling of musical instruments for purposes of sound synthesis is reviewed. Recent references, results, and outstanding problems are highlighted for models of strings, winds, brasses, percussion, and acoustic spaces. Emphasis is placed on digital waveguide models and the musical acoustics research on which they are based.

We describe the attempt to synthesize emotional speech with a concatenative speech synthesizer using a parameter space covering not only f0, duration and amplitude, but also voice quality parameters, spectral energy distribution, harmonics-to-noise ratio, and articulatory precision. The application of these extended parameter set offers the possibility to combine the high segmental quality of concatenative synthesis with a wider range of control settings needed for the synthesis of natural affected speech. 1 INTRODUCTION The quality of synthesized speech is usually measured in terms of intelligibility and naturalness. State of the art synthesizers are well intelligible under regular conditions. Therefore, development has concentrated on the improvement of naturalness. An obvious way to follow is the incorporation of correct and functionally adequate prosody. Another maybe less obvious but very interesting aim is the generation of non-neutral affect. For the English language a system f...

Although sinusoidal models have been demonstrated to be capable of high-quality musical instrument synthesis [1], speech modification [2], and speech synthesis [3], little exploration of the application of these models to the synthesis of singing voice has been undertaken. In this paper, we propose a system framework similar to that employed in concatenation-based text-to-speech synthesizers, and describe its extension to the synthesis of singing voice. The power and flexibility of the sinusoidal model used in the waveform synthesis portion of the system [1] enables high-quality, computationally-efficient synthesis and the incorporation of musical qualities such as vibrato and spectral tilt variation. Modeling of segmental phonetic characteristics is achieved by employing a "unit selection" procedure that selects sinusoidally-modeled segments from an inventory of singing voice data collected from a human vocalist. The system, called Lyricos, is capable of synthesizing very natural-soun...

In this paper, we propose a system for synthesizing the human singing voice and the musical subtleties that accompany it. The system, Lyricos, employs a concatenation-based text-to-speech method to synthesize arbitrary lyrics in a given language. Using information contained in a regular MIDI file, the system chooses units, represented as sinusoidal waveform model parameters, from an inventory of data collected from a professional singer, and concatenates these to form arbitrary lyrical phrases. Standard MIDI messages control parameters for the addition of vibrato, spectral tilt, and dynamic musical expression, resulting in a very natural-sounding singing voice.

This paper reviews the popular methods and models used for the synthesis of the singing voice, discussing strengths and weaknesses of each technique. Then a brief review is given of research on cross-modal visual/auditory perception of the human voice. The paper concludes with comments related to the singing synthesis systems discussed, addressing multi-modal perception, audio morphing, and the categorical perception of sound. 1 Introduction The human voice is the most ubiquitous, flexible, and general of acoustic instruments. We all have one, yet only a few of us learn to "play" it with proficiency as a musical instrument. Most functions of this instrument we take for granted, but huge regions of our brains are dedicated to controlling and perceiving the sounds made by it. Even those people that never learn to use it musically still are able to perform amazing feats of imitation and flexibility with their voice. The voice can exact independent control across a broad range of pit...

A wide variety of singing synthesis models and methods exist, but there are remarkably few real-time controllers for these models. This paper describes a variety of devices developed over the last few years for controlling singing synthesis models implemented in the Synthesis Toolkit in C++ (STK), Max/MSP, and ChucK. All of the controllers share some common features, such as air-pressure sensing for breathing and/or loudness control, means to control pitch, and methods for selecting and blending phonemes, diphones, and words. However, the form factors, sensors, mappings, and algorithms vary greatly between the different controllers.

We describe the implementation of an environment for Gesturally-Realized Audio, Speech and Song Performance (GRASSP), which includes a glove-based interface, a mapping/training interface, and a collection of Max/MSP/Jitter bpatchers that allow the user to improvise speech, song, sound synthesis, sound processing, sound localization, and video processing. The mapping/training interface provides a framework for performers to specify by example the mapping between gesture and sound or video controls. We demonstrate the effectiveness of the GRASSP environment for gestural control of musical expression by creating a gesture-to-voice system that is currently being used by performers.

We present an overview of digital audio analysis and synthesis methods for sound design and composition. The sonic landscape available to us contains a multitude of sounds, ranging from artificial to natural, purely musical to purely “real-world. ” To take full advantage of this diversity, it is helpful to have a comprehensive knowledge of the tools with which we can create and manipulate different types of sounds. We offer a summary of existing techniques organized by their underlying technology and source material, as well as by the kinds of sounds for which they are known to be effective. Our survey aims to support the synthesis of rich sound scenes and environments by facilitating the selection of the most appropriate tools for each component sound. A full toolbox means the whole world need not look like a nail! 1.

In this paper, we present the first step of a project that is able to perform both speech and singing synthesis controlled in real-time. Our aim is to develop a flexible application allowing performers to produce complex and versatile singing as well as speech. Thus, we have adapted an existing speech synthesizer, the MBROLA software, to real-time singing constraints. The work presented in this paper concerns the integration of the MBROLA speech synthesizer into the Max/MSP real-time environment through the development of an external object. We present real-time control functions of the object and discuss its limitations. We further discuss about perspectives in the development of MBROLA-based computer music applications. 1.