This paper first introduces two previous software-based music instruments designed by the author, and analyses the crucial importance of the visual feedback introduced by their interfaces. A quick taxonomy and analysis of the visual components in current trends of interactive music software is then proposed, before introducing the reacTable*, a new project that is currently under development. The reacTable* is a collaborative music instrument, aimed both at novices and advanced musicians, which employs computer vision and tangible interfaces technologies, and pushes further the visual feedback interface ideas and techniques aforementioned.

This article reflects the current state of the reacTable* project, an electronic music instrument with a tangible table-based interface, which is currently under development at the Audiovisual Institute at the Universitat Pompeu Fabra. In this paper we are focussing on the issue of Dynamic Patching, which is a particular and unique aspect of the sound synthesis and control paradigms of the reacTable*. Unlike common visual programming languages for sound synthesis, which conceptually separate the patch building process from the actual musical performance, the reacTable* combines the construction and playing of the instrument in a unique way.

This paper describes the design of an expressive tangible interface for cinema editing as a live performance. A short survey of live video practices is provided. The Live Cinema instrument is a cross between a musical instrument and a film editing tool, tailored for improvisational control as well as performance presence. Design specifications for the instrument evolved based on several types of observations including: our own performances in which we used a prototype based on available tools; an analysis of performative aspects of contemporary DJ equipment; and an evaluation of organizational aspects of several generations of film editing tools. Our instrument presents the performer with a large canvas where projected images can be grabbed and moved around with both hands simultaneously; the performer also has access to two video drums featuring haptic display to manipulate the shots and cut between streams. The paper ends with a discussion of issues related to the tensions between narrative structure and hands-on control, live and recorded arts and the scoring of improvised films.

We report on The Manual Input Sessions, a series of audiovisual vignettes which probe the expressive possibilities of free-form hand gestures. Performed on a hybrid projection system which combines a traditional analog overhead projector and a digital PC video projector, our vision-based software instruments generate dynamic sounds and graphics solely in response to the forms and movements of the silhouette contours of the user’s hands. Interactions and audiovisual mappings which make use of both positive (exterior) and negative (interior) contours are discussed.

Introduction Joseph A. Paradiso Responsive Environments Group MIT Media Laboratory 77 Massachusetts Avenue, NE18-5F Cambridge, Massachusetts 02142 USA joep@media.mit.edu Sile O'Modhrain Palpable Machines Group Media Lab Europe Sugar House Lane, Bellevue Dublin 8, Ireland sile@media.mit.edu 1) History and Evolution of Musical Controllers Throughout history, each set of technologies, from woodcraft to water pumps and from electricity to computers, has ushered its own set of changes into the way people generate and interact with music. Acoustic musical instruments have settled into canonical forms, taking centuries, if not millennia, to evolve their balance between sound production, ergonomics, playability, potential for expression, and aesthetic design. In contrast, electronic instruments have been around for little more than a century, during which rapid, often exponential (Kurzweil, 2000) advances in technology have continually opened new possibilities for sound synthesis

In this paper, after giving some possible definitions for expressiveness, we examine the problem of expressiveness in digital musical instruments, which tends to involve using specific gestures to obtain an expressive sound rather than performing expressive gestures. Some of the particular features of digital musical instruments, such as pitch control, dynamic control and the possibility of exploring sound palettes, are described and some practical examples given. Lastly, several musical implications of the gestures used to obtain musical expressiveness are discussed, from the pedagogical and other related points of view. 1.

This paper gives an introduction into the principles of interactivity in music video games. Music video games are an old but small genre of games. The earliest direct ancestors emerged in the 1970ies. Some recent music video games were hugely successful. Until today, there are only a few different approaches to their design. The purpose of this article is to shed light on what these design principles are, and how the player is immersed. By analysing several games qualitatively, we extracted certain typical features of games of this genre: active scores, rhythm action, quantisation, synaesthesia, play as performance, free-form play, and sound agents. All these aspects of music video games are discussed in this paper with the aim of describing how they affect the interactivity of the games. The result is a grammar of the language of music video games. Linked to adequate metaphors, this grammar can build a veritable repository for rhythm based, melodically interactive games and digital electronic instruments.

While personal computers have been used for over twenty years in live music contexts, the proliferation of powerful and affordable portable systems (laptops) has spurred the emergence of new music creation styles and venues outside of academia. Laptop performance is the practice of live computer music using software tools exclusively. This paper surveys the current state of laptop performance practice, discussing the styles of music that are typically performed in this way, and the techniques and tools used. This discussion serves as a context for a new software system for laptop performance. The system’s design is motivated by some of the problems encountered in this style of live use. The system supports the real-time definition and modification of generative musical patterns via a novel freehand drawing interface, allowing a solo performer to create multi-layered works on-stage. 1

This paper begins by evaluating various systems in terms of factors for building interactive audiovisual environments. The main issues for flexibility and expressiveness in the generation of dynamic sounds and images are then isolated. The design and development of an audiovisual system prototype is described at the end.

Figure 0. Multiple Bouncing Spheres interfaces visualized from a centralized viewpoint. Each human player manipulates spheres over a portion of the squares. The ensemble is synchronized by computer. Emergence is the formation of complex patterns from simpler rules or systems. This paper motivates and describes new graphical interfaces for controlling sound designed for strongly-timed, collaborative computer music ensembles. While the interfaces are themselves minimal and often limiting, the overall collaboration can produce results novel beyond the simple sum of the components – leveraging the very uniqueness of an ensemble: its strength in numbers. The interfaces are human-controlled and machine-synchronized across a dozen or more computers. Group control, as well as sound synthesis mapping at each endpoint, can be programmed quickly and even on-the-fly, providing a second channel of real-time control. We show examples of these interfaces as interchangeable plug-ins for the Audicle environment, and also document how they are used in a laptop ensemble. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Nime’06, June 4-8, 2006, Paris, France. Copyright remains with the author(s).