Animations for which sound effects were automatically added by our system, demonstrated in the accompanying video. (a) A real wok in which a pebble is thrown; the pebble rattles around the wok and comes to rest after wobbling. (b) A simulation of a pebble thrown in wok, with all sound effects automatically generated. (c) A ball rolling back and forth on a ribbed surface. (d) Interaction with a sonified object. We describe algorithms for real-time synthesis of realistic sound effects for interactive simulations (e.g., games) and animation. These sound effects are produced automatically, from 3D models using dynamic simulation and user interaction. We develop algorithms that are efficient, physicallybased, and can be controlled by users in natural ways. We develop effective techniques for producing high quality continuous contact sounds from dynamic simulations running at video rates which are slow relative to audio synthesis. We accomplish this using modal models driven by contact forces modeled at audio rates, which are much higher than the graphics frame rate. The contact forces can be computed from simulations or can be custom designed. We demonstrate the effectiveness with complex realistic simulations.

Figure 1: Left, an overview of a test virtual environment, containing 174 sound sources. All vehicles are moving. Mid-left, the magenta dots indicate the locations of the sound sources while the red sphere represents the listener. Notice that the train and the river are extended sources modeled by collections of point sources. Mid-right, ray-paths from the sources to the listener. Paths in red correspond to the perceptually masked sound sources. Right, the blue boxes are clusters of sound sources with the representatives of each cluster in grey. Combination of auditory culling and spatial clustering allows us to render such complex audio-visual scenes in real-time. We propose a real-time 3D audio rendering pipeline for complex virtual scenes containing hundreds of moving sound sources. The approach, based on auditory culling and spatial level-of-detail, can handle more than ten times the number of sources commonly available on consumer 3D audio hardware, with minimal decrease in audio quality. The method performs well for both indoor and outdoor environments. It leverages the limited capabilities of audio hardware for many applications, including interactive architectural acoustics simulations and automatic 3D voice management for video games. Our approach dynamically eliminates inaudible sources and groups the remaining audible sources into a budget number of clusters. Each cluster is represented by one impostor sound source, positioned using perceptual criteria. Spatial audio processing is then performed only on the impostor sound sources rather than on every original source thus greatly reducing the computational cost. A pilot validation study shows that degradation in audio quality, as well as localization impairment, are limited and do not seem to vary significantly with the cluster budget. We conclude that our real-time perceptual audio rendering pipeline can generate spatialized audio for complex auditory environments without introducing disturbing changes in the resulting perceived soundfield.

and compelling sounds that correspond to the motions of rigid objects. By numerically precomputing the shape and frequencies of an object's deformation modes, audio can be synthesized interactively directly from the force data generated by a standard rigid-body simulation. Using sparse-matrix eigen-decomposition methods, the deformation modes can be computed efficiently even for large meshes. This approach allows us to accurately model the sounds generated by arbitrarily shaped objects based only on a geometric description of the objects and a handful of material parameters. We validate our method by comparing results from a simulated set of wind chimes to audio measurements taken from a real set.

A difficult challenge in geometrical acoustic modeling is computing propagation paths from sound sources to receivers fast enough for interactive applications. We paper describe a beam tracing method that enables interactive updates of propagation paths from a stationary source to a moving receiver. During a precomputation phase, we trace convex polyhedral beams from the location of each sound source, constructing a "beam tree" representing the regions of space reachable by potential sequences of transmissions, diffractions, and specular reflections at surfaces of a 3D polygonal model. Then, during an interactive phase, we use the precomputed beam trees to generate propagation paths from the source(s) to any receiver location at interactive rates. The key features of our beam tracing method are: 1) it scales to support large architectural environments, 2) it models propagation due to wedge diffraction, 3) it finds all propagation paths up to a given termination criterion without exhaustive search or risk of under-sampling, and 4) it updates propagation paths at interactive rates. We demonstrate use of this method for interactive acoustic design of architectural environments.

Many interactive applications, such as video games, require processing a large number of sound signals in real-time. This paper proposes a novel perceptually-based and scalable approach for efficiently filtering and mixing a large number of audio signals. Key to its efficiency is a pre-computed Fourier frequency-domain representation augmented with additional descriptors. The descriptors can be used during the real-time processing to estimate which signals are not going to contribute to the final mixture. Besides, we also propose an importance sampling strategy allowing to tune the processing load relative to the quality of the output. We demonstrate our approach for a variety of applications including equalization and mixing, reverberation processing and spatialization. It can also be used to optimize audio data streaming or decompression. By reducing the number of operations and limiting bus traffic, our approach yields a 3 to 15-fold improvement in overall processing rate compared to brute-force techniques, with minimal degradation of the output. 1.

Traditional uses of virtual audio environments tend to focus on perceptually accurate acoustic representations. Though spatialization of sound sources is important, it is necessary to leverage control of the sonic representation when considering musical applications. The proposed framework allows for the creation of perceptually immersive scenes that function as musical instruments. Loudspeakers and microphones are modeled within the scene along with the listener/performer, creating a navigable 3D sonic space where sound sources and sinks process audio accordingtouser-defined spatial mappings.

An image method due to Allen and Berkley (1979) is often used to simulate the effect of reverberation in rooms. This method is relatively expensive computationally. We present a fast method for conducting such simulations using multipole expansions. For M real and image sources and N evaluation points, while the image method requires O(MN) operations, our method achieves the calculations in O(M + N) operations, resulting in a substantial speedup. Applications of our technique are also expected in simulation of virtual audio.

This paper describes and analyzes a beam tracing method for computing sound propagation paths from sound sources to receivers in architectural environments. The algorithm traces polyhedral beams from the location of each sound source through a precomputed data structure encoding spatial adjacencies. The main features of the method are: 1) it enumerates all potential sequences of specular reflection, diffraction, and transmission up to user-specified termination criteria, 2) it enables evaluation of early propagation paths at interactive rates, and 3) it scales to support large, "densely-occluded" architectural environments. It is being used to support real-time auralization in immersive virtual environment applications. 1.

��×� � ÓÒ Ø� � �Ñ�� � ×ÓÙÖ � Ñ�Ø�Ó � �ÓÖ �ÙÖ�Ð�Þ�Ø�ÓÒ ÔÙÖÔÓ×� × � ÐÓÛ ÓÖ��Ö ����Ø�Ð ¬ÐØ�Ö �ÓÖ � � � ��«Ö � Ø�Ò � ��� � �×

We present an interactive algorithm to compute sound propagation paths for transmission, specular reflection and edge diffraction in complex scenes. Our formulation uses an adaptive frustum representation that is automatically sub-divided to accurately compute intersections with the scene primitives. We describe a simple and fast algorithm to approximate the visible surface for each frustum and generate new frusta based on specular reflection and edge diffraction. Our approach is applicable to all triangulated models and we demonstrate its performance on architectural and outdoor models with tens or hundreds of thousands of triangles and moving objects. In practice, our algorithm can perform geometric sound propagation in complex scenes at 4-20 frames per second on a multi-core PC.