This thesis presents a new computer interface metaphor for the real-time and simultaneous performance of dynamic imagery and sound. This metaphor is based on the idea of an inexhaustible, infinitely variable, time-based, audiovisual “substance ” which can be gesturally created, deposited, manipulated and deleted in a free-form, non-diagrammatic image space. The interface metaphor is exemplified by five interactive audiovisual synthesis systems whose visual and aural dimensions are deeply plastic, commensurately malleable, and tightly connected by perceptuallymotivated mappings. The principles, patterns and challenges which structured the design of these five software systems are extracted and discussed, after which the expressive capacities of the five systems are compared and evaluated.

This article describes experiments conducted with the system and 3- to 5-year-old children. We highlight several dimensions of the study pertaining to music education, including attention span, spontaneous development of playing modes, and capacity to listen analytically. We describe very encouraging preliminary results and stress the importance of using reflective interactive systems for triggering musical interest in children and creating stimulating, nonsupervised music learning environments. We conclude by setting up our research in the context of the theory of flow as an optimal experience.

The design of tools for creative activities affects the creative processes and output of users. In this paper we consider how an understanding of creative interaction can inform the design of support tools in a creative domain, and where creative needs cross domain boundaries. Using observations of musical composers we analyse the theoretical approaches to understanding creativity and their use to HCI. Cycles of ideation and evaluation are suggested as atomic elements of creative interactions, with the representation of ideas a central activity for individual and collaborating composers. A model of collaborative composition was developed, along with an analysis of the representational types used in the domain. This led to the design and evaluation of a prototype Sonic Sketchpad for musical idea representation.