This paper argues for a special focus on the use of dynamic human interaction to explore datasets while they are being transformed into sound. We describe why this is a special case of both human computer interaction (HCI) techniques and sonification methods. Humans are adapted for interacting with their physical environment and making continuous use of all their senses. When this exploratory interaction is applied to a dataset (by continuously controlling its transformation into sound) new insights are gained into the data’s macro and micro-structure, which are not obvious in a visual rendering. This paper reviews the importance of interaction in sonification, describes how a certain quality of interaction is required, provides examples of the techniques being applied interactively, and outlines a plan of future work to develop interaction techniques to aid sonification. 1.

Shoogle is a novel, intuitive interface for sensing data within a mobile device, such as presence and properties of text messages or remaining resources. It is based around active exploration: devices are shaken, revealing the contents rattling around “inside”. Vibrotactile display and realistic impact sonification create a compelling system. Inertial sensing is used for completely eyes-free, single-handed interaction that is entirely natural. Prototypes are described running both on a PDA and on a mobile phone with a wireless sensor pack. Scenarios of use are explored where active sensing is more appropriate than the dominant alert paradigm. 1

This paper describes a new approach to render sonifications for high-dimensional data, allowing the user to perceive the “main” structure of the data distribution. This is achieved by computing the principal curve of the data set, which is a trajectory that passes through the “middle ” of the data and allows to define a time order on the data points. The sonification can be imagined as the time-variant auditory scene, perceived while moving along the principal curve. In this paper a method for computing principal curves is presented, the sonification concept is introduced and some sonification examples are given.

Abstract: We argue that the direct experimental approaches to elucidate the architecture of higher brains may benefit from insights gained from exploring the possibilities and limits of artificial control architectures for robot systems. We present some of our recent work that has been motivated by that view and that is centered around the study of various aspects of hand actions since these are intimately linked with many higher cognitive abilities. As examples, we report on the development of a modular system for the recognition of continuous hand postures based on neural nets, the use of vision and tactile sensing for guiding prehensile movements of a multifingered hand, and the recognition and use of hand gestures for robot teaching. Regarding the issue of learning, we propose to view real-world learning from the perspective of data mining and to focus more strongly on the imitation of observed actions instead of purely reinforcement-based exploration. As a concrete example of such an effort we report on the status of an ongoing project in our lab in which a robot equipped with an attention system with

Markov chain Monte Carlo (McMC) simulation is a popular computational tool for making inferences from complex, high-dimensional probability densities. Given a particular target density p,the idea behind this technique is to simulate a Markov chain that has p as its stationary distribution. To be successful, the chain needs to be run long enough so that the distribution of the current draw is close to the target density. Unfortunately, very few diagnostic tools exist to monitor characteristics of the chain. In this paper, we present a new approach to render sonifications of McMC simulations. The proposed method consists of several auditory streams which provide information about the behavior of the Markov chain. In particular, we focus on uncovering modes in the target density function. In addition to monitoring, we have found our sonification to be an effective means for understanding the structure of high-dimensional densities. We have also applied our method to the exploratory analysis of highdimensional data sets. In this case, we take as our target p a nonparametric density estimate obtained from the data. In this paper, we present a detailed description of our sonification design and illustrate its performance on test cases consisting of both synthetic and real-world data sets. Sound examples are also given. 1.

This paper introduces Crystallization Sonification, a sonification model for exploratory analysis of high-dimensional datasets. The model is designed to provide information about the intrinsic data dimensionality (which is a local feature) and the global data dimensionality, as well as the transitions between a local and global view on a dataset. Furthermore the sound allows to display the clustering in high-dimensional datasets. The model defines a crystal growth process in the high-dimensional data-space which starts at a user selected “condensation nucleus ” and incrementally includes neighboring data according to some growth criterion. The sound summarizes the temporal evolution of this crystal growth process. For introducing the model, a simple growth law is used. Other growth laws which are used in the context of hierarchical clustering are also suited and their application in crystallization sonification offers new ways to inspect the results of data clustering as an alternative to dendrogram plots. In this paper, the sonification model is described and example sonifications are presented for some synthetic high-dimensional datasets. 1.

This paper presents a new sonification model for the exploration of topographically ordered high-dimensional data (multi-parameter maps, volume data) where each data item consists of a position and feature vector. The sonification model implements a common metaphor from thermodynamics that heat can be interpreted as stochastic motion of ’molecules’. The latter are determined by the data under examination, and ’live ’ only in the feature space. Heat-induced interactions cause acoustic events that fuse to a granular sound texture which conveys meaningful information about the underlying distribution in feature space. As a second ingredient of the model, data selection is achieved by a separated navigation process in position space using a dynamic aura model, such that heat can be induced locally. Both, a visual and an auditory display are driven by the underlying model. We exemplify the sonification by means of interaction examples for different high-dimensional distributions.

Abstract—We investigate the use of audio and haptic feedback to augment the display of a mobile device controlled by tilt input. We provide an example of this based on Doppler effects, which highlight the user’s approach to a target, or a target’s movement from the current state, in the same way we hear the pitch of a siren change as it passes us. Twelve participants practiced navigation/browsing a state-space that was displayed via audio and vibrotactile modalities. We implemented the experiment on a Pocket PC, with an accelerometer attached to the serial port and a headset attached to audio port. Users navigated through the environment by tilting the device. Feedback was provided by audio displayed via a headset, and by vibrotactile information displayed via a vibrotactile unit in the Pocket PC. Users selected targets placed randomly in the state-space, supported by combinations of audio, visual and vibrotactile cues. The speed of target acquisition and error rate were measured, and summary statistics on the acquisition trajectories were calculated. These data were used to compare different display combinations and configurations. The results in the paper quantified the changes brought by predictive or ‘quickened ’ sonified displays in mobile, gestural interaction. Index Terms—Auditory interfaces, Doppler effect, haptic feedback, display quickening, handheld devices, accelerometer

We describe an excitation interface for displaying data on mobile devices, based around active exploration: devices are shaken, revealing the contents rattling around inside. This combines sample-based contact sonification with event-playback vibrotactile feedback for a rich and compelling display. Motion is sensed from accelerometers, directly linking the motions of the user to the feedback they receive in a tightly-closed loop. The resulting interface requires no visual attention, and can be operated blindly with a single hand: it is reactive rather than disruptive. This interaction style is applied to the display of an SMS inbox. We use language models to extract salient features from text messages automatically. The output of this classification process controls the timbre and physical dynamics of the simulated objects. The interface gives a rapid semantic overview of the contents of an inbox, without compromising privacy or interrupting the user.

Abstract. Data Mining in Virtual Reality (VR) has, for good reasons, been focused on facilitating visual inspection and analysis. As the amount and complexity of data is overwhelming, it is worthwhile to consider a further exploration of the human perceptual faculties, and it seems natural to consider sound as a possible perceptual cue. Current technology enables us to create advanced real-time 3D soundscapes, which may prove useful since the human ears ’ field of hearing is larger than the eyes ’ field of view, and thus is able to inform us on events happening in areas that we do no see at a given time. This paper presents methods for generating 3D soundscapes from statistical information. Through a series of investigations, we present and discuss the effectiveness of these methods in situations where sound acts as support for visual cues, as well as the use of sound as a separate cue for analyzing data in VR. 1