Results 1 - 10
of
22
A Machine Learning Toolbox for Musician Computer Interaction
- In Proc. NIME 2011
, 2011
"... This paper presents the SARC EyesWeb Catalog, (SEC), a machine learning toolbox that has been specifically developed for musician-computer interaction. The SEC features a large number of machine learning algorithms that can be used in real-time to recognise static postures, perform regression and cl ..."
Abstract
-
Cited by 12 (4 self)
- Add to MetaCart
(Show Context)
This paper presents the SARC EyesWeb Catalog, (SEC), a machine learning toolbox that has been specifically developed for musician-computer interaction. The SEC features a large number of machine learning algorithms that can be used in real-time to recognise static postures, perform regression and classify multivariate temporal gestures. The algorithms within the toolbox have been designed to work with any N-dimensional signal and can be quickly trained with a small number of training examples. We also provide the motivation for the algorithms used for the recognition of musical gestures to achieve a low intra-personal generalisation error, as opposed to the inter-personal generalisation error that is more common in other areas of humancomputer interaction.
Human model evaluation in interactive supervised learning
- Proc. CHI‘10
, 2010
"... Model evaluation plays a special role in interactive machine learning (IML) systems in which users rely on their assessment of a model’s performance in order to determine how to improve it. A better understanding of what model criteria are important to users can therefore inform the design of user i ..."
Abstract
-
Cited by 8 (1 self)
- Add to MetaCart
(Show Context)
Model evaluation plays a special role in interactive machine learning (IML) systems in which users rely on their assessment of a model’s performance in order to determine how to improve it. A better understanding of what model criteria are important to users can therefore inform the design of user interfaces for model evaluation as well as the choice and design of learning algorithms. We present work studying the evaluation practices of end users interactively building supervised learning systems for real-world gesture analysis problems. We examine users ’ model evaluation criteria, which span conventionally relevant criteria such as accuracy and cost, as well as novel criteria such as unexpectedness. We observed that users employed evaluation techniques— including cross-validation and direct, real-time evaluation— not only to make relevant judgments of algorithms ’ performance and interactively improve the trained models, but also to learn to provide more effective training data. Furthermore, we observed that evaluation taught users about what types of models were easy or possible to build, and users sometimes used this information to modify the learning problem definition or their plans for using the trained models in practice. We discuss the implications of these findings with regard to the role of generalization accuracy in IML, the design of new algorithms and interfaces, and the scope of potential benefits of incorporating human interaction in the design of supervised learning systems.
A Hierarchical Approach for the Design of Gesture–to
- Sound Mappings”, Proceedings of the 9th Sound and Music Conference (SMC), Copenhague, Danemark
, 2012
"... We propose a hierarchical approach for the design of gesture-to-sound mappings, with the goal to take into account multilevel time structures in both gesture and sound processes. This allows for the integration of temporal mapping strategies, complementing mapping systems based on instantaneous rela ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
We propose a hierarchical approach for the design of gesture-to-sound mappings, with the goal to take into account multilevel time structures in both gesture and sound processes. This allows for the integration of temporal mapping strategies, complementing mapping systems based on instantaneous relationships between gesture and sound synthesis parameters. As an example, we propose the implementation of Hierarchical Hidden Markov Models to model gesture input, with a flexible structure that can be authored by the user. Moreover, some parameters can be adjusted through a learning phase. We show some examples of gesture segmentations based on this approach, considering several phases such as preparation, attack, sustain, release. Finally we describe an application, developed in Max/MSP, illustrating the use of accelerometer-based sensors to control phase vocoder synthesis techniques based on this approach. 1.
A Methodological Framework for Teaching, Evaluating and Informing NIME Design with a Focus on Expressiveness and Mapping
- In NIME ’14 Proceedings of the 2014 Conference on New Interfaces for Musical Expression
, 2014
"... ABSTRACT The maturation process of the NIME field has brought a growing interest in teaching the design and implementation of Digital Music Instruments (DMIs) as well as in finding objective evaluation methods to assess the suitability of these outcomes. In this paper we propose a methodology for t ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
ABSTRACT The maturation process of the NIME field has brought a growing interest in teaching the design and implementation of Digital Music Instruments (DMIs) as well as in finding objective evaluation methods to assess the suitability of these outcomes. In this paper we propose a methodology for teaching NIME design and a set of tools meant to inform the design process. This approach has been applied in a master course focused on the exploration of expressiveness and on the role of the mapping component in the NIME creation chain, through hands-on and self-reflective approach based on a restrictive setup consisting of smart-phones and the Pd programming language. Working Groups were formed, and a 2-step DMI design process was applied, including 2 performance stages. The evaluation tools assessed both System and Performance aspects of each project, according to Listeners' impressions after each performance. Listeners' previous music knowledge was also considered. Through this methodology, students with different backgrounds were able to effectively engage in the NIME design processes, developing working DMI prototypes according to the demanded requirements; the assessment tools proved to be consistent for evaluating NIMEs systems and performances, and the fact of informing the design processes with the outcome of the evaluation, showed a traceable progress in the students' outcomes.
MIXPLORATION: Rethinking the Audio Mixer Interface
"... A typical audio mixer interface consists of faders and knobs that control the amplitude level as well as processing (e.g. equalization, compression and reverberation) parameters of individual tracks. This interface, while widely used and effec-tive for optimizing a mix, may not be the best interface ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
A typical audio mixer interface consists of faders and knobs that control the amplitude level as well as processing (e.g. equalization, compression and reverberation) parameters of individual tracks. This interface, while widely used and effec-tive for optimizing a mix, may not be the best interface to fa-cilitate exploration of different mixing options. In this work, we rethink the mixer interface, describing an alternative inter-face for exploring the space of possible mixes of four audio tracks. In a user study with 24 participants, we compared the effectiveness of this interface to the traditional paradigm for exploring alternative mixes. In the study, users responded that the proposed alternative interface facilitated exploration and that they considered the process of rating mixes to be benefi-cial. Author Keywords Audio; music; mixing; exploratory interfaces
A voice interface for sound generators: adaptive and automatic mapping of gestures to sound
- In Proc. of NIME
, 2012
"... ABSTRACT Sound generators and synthesis engines expose a large set of parameters, allowing run-time timbre morphing and exploration of sonic space. However, control over these high-dimensional interfaces is constrained by the physical limitations of performers. In this paper we propose the exploita ..."
Abstract
-
Cited by 2 (1 self)
- Add to MetaCart
(Show Context)
ABSTRACT Sound generators and synthesis engines expose a large set of parameters, allowing run-time timbre morphing and exploration of sonic space. However, control over these high-dimensional interfaces is constrained by the physical limitations of performers. In this paper we propose the exploitation of vocal gesture as an extension or alternative to traditional physical controllers. The approach uses dynamic aspects of vocal sound to control variations in the timbre of the synthesized sound. The mapping from vocal to synthesis parameters is automatically adapted to information extracted from vocal examples as well as to the relationship between parameters and timbre within the synthesizer. The mapping strategy aims to maximize the breadth of the explorable perceptual sonic space over a set of the synthesizer's real-valued parameters, indirectly driven by the voice-controlled interface.
Wekinating 000000Swan: Using Machine Learning to Create and Control Complex Artistic Systems
, 2011
"... In this paper we discuss how the band 000000Swan uses machine learning to parse complex sensor data and create intricate artistic systems for live performance. Using the Wekinator software for interactive machine learning, we have created discrete and continuous models for controlling audio and visu ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
In this paper we discuss how the band 000000Swan uses machine learning to parse complex sensor data and create intricate artistic systems for live performance. Using the Wekinator software for interactive machine learning, we have created discrete and continuous models for controlling audio and visual environments using human gestures sensed by a commercially-available sensor bow and the Microsoft Kinect. In particular, we have employed machine learning to quickly and easily prototype complex relationships between performer gesture and performative outcome.
Cognitive Architecture in Mobile Music Interactions ABSTRACT
, 2011
"... This paper explores how a general cognitive architecture can pragmatically facilitate the development and exploration of interactive music interfaces on a mobile platform. To this end we integrated the Soar cognitive architecture into the mobile music meta-environment urMus. We develop and demonstra ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
This paper explores how a general cognitive architecture can pragmatically facilitate the development and exploration of interactive music interfaces on a mobile platform. To this end we integrated the Soar cognitive architecture into the mobile music meta-environment urMus. We develop and demonstrate four artificial agents which use diverse learning mechanisms within two mobile music interfaces. We also include details of the computational performance of these agents, evincing that the architecture can support real-time interactivity on modern commodity hardware.
2011. “A demonstration of bow articulation recognition with Wekinator and KBow
- Proc. International Computer Music Conference
"... Using the Wekinator software tool for real-time, interactive machine learning [3] and the K-Bow commercial sensor bow [5], we have constructed a realtime cello bow articulation classification system. This system is capable of outputting articulation labels (e.g., “legato, ” “marcato, ” “spiccato”) i ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Using the Wekinator software tool for real-time, interactive machine learning [3] and the K-Bow commercial sensor bow [5], we have constructed a realtime cello bow articulation classification system. This system is capable of outputting articulation labels (e.g., “legato, ” “marcato, ” “spiccato”) in real-time as a cellist performs. These labels, which are output via Open Sound Control [9], may be used in conjunction with visualization or music tools in composition and live performance. Our work is distinguished from prior work in bow gesture recognition in that the Wekinator allows a musician user to rapidly build customized bow gesture models from scratch by demonstrating bowing gestures to form a training set; the user can also interactively refine these models through iterative changes to both the learning algorithms and dataset. In this paper, we briefly describe our work creating articulation models for our own use. In particular, we show that the Wekinator and K-Bow together allowed for the fast creation of accurate models. We then propose a hands-on demonstration of this work in which ICMC attendees can use the K-Bow to interactively build their own gesture classifiers. 1.
Exploring Reinforcement Learning for Mobile Percussive Collaboration ABSTRACT
"... This paper presents a system for mobile percussive collaboration. We show that reinforcement learning can incrementally learn percussive beat patterns played by humans and supports real-time collaborative performance in the absence of one or more performers. This work leverages an existing integrati ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
This paper presents a system for mobile percussive collaboration. We show that reinforcement learning can incrementally learn percussive beat patterns played by humans and supports real-time collaborative performance in the absence of one or more performers. This work leverages an existing integration between urMus and Soar and addresses multiple challenges involved in the deployment of machine-learning algorithms for mobile music expression, including tradeoffs between learning speed & quality; interface design for human collaborators; and real-time performance and improvisation.
