See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228703247

Ray acoustics using computer graphics technology

Article

CITATIONS READS

50 2,163

3 authors, including:

Niklas Röber Maic Masuch

NVIDIA University of Duisburg-Essen
46 PUBLICATIONS 1,574 CITATIONS 150 PUBLICATIONS 2,433 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Niklas Röber on 10 December 2014.

The user has requested enhancement of the downloaded file.

Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

RAY ACOUSTICS USING COMPUTER GRAPHICS TECHNOLOGY

Niklas Röber, Ulrich Kaminski and Maic Masuch

Games Research Group
Department of Simulation and Graphics,
Otto-von-Guericke-University Magdeburg, Germany
niklas@isg.cs.uni-magdeburg.de

ABSTRACT very well suited for the computation of parallel problems and was
The modeling of room acoustics and simulation of sound wave also more recently employed as DSP for sound signal processing
propagation remain a difficult and computationally expensive task. [2, 3]. In the area of sound simulations, the GPU was used to solve
Two main techniques have evolved, with one focusing on a real basic geometric room acoustics [4], as well as wave-based sound
physical - wave-oriented - sound propagation, while the other ap- propagation using waveguide meshes [5]. Besides some physi-
proximates sound waves as rays using raytracing techniques. Due cal differences, the propagation of sound and light share several
to many advances in computer science, and especially computer similarities that make existing graphics rendering techniques ex-
graphics over the last decade, interactive 3D sound simulations for ploitable to accommodate an acoustic energy propagation model.
complex and dynamic environments are within reach. The goal of this work is to build a foundation for ray-based
In this paper we analyze sound propagation in terms of acoustic sound simulations using an acoustic energy propagation model,
energy and explore the possibilities to map these concepts to ra- and furthermore, to demonstrate its applicability and efficiency
diometry and graphics rendering equations. Although we concen- using modern graphics hardware and rendering techniques. We
trate on ray-based techniques, we also partially consider wave- derive the acoustic rendering equations from global illumination
based sound propagation effects. The implemented system ex- models and radiometry used in computer graphics [6], and extend
ploits modern graphics hardware and rendering techniques and is the existing model by time- and frequency dependencies. This par-
able to efficiently simulate 3D room acoustics, as well as to mea- adigm is later employed in a GPU-based implementation to per-
sure simplified personal HRTFs through acoustic raytracing. form realtime sound simulations using ray-based techniques for
the applications of room acoustics and personalized HRTF simu-
lations. The audible spectrum is divided into 10 frequency bands,
1. INTRODUCTION
which are interpreted individually with respect to their wavelength
and energy. The local energy contribution of each surface patch is
Physically correct sound simulations of larger and more complex
evaluated separately per frequency band using functions of reflec-
environments remain a difficult, if not impossible task. This is
tion, transmission/refraction, absorption and diffraction. Finally,
mainly due to the extensive nature of sound wave propagation,
the acoustic energy at the observers position is accumulated and
along its complex interaction with scene objects. Unlike light,
filtered regarding direction and distance using HRTFs. The sys-
the audible spectrum covers a large area of frequency bands (oc-
tem allows us to simulate realtime interactive and dynamic envi-
taves), and is additionally, due to a slow propagation, highly time-
ronments with varying acoustic materials, but also to approximate
dependent. Although, this introduces several complications, it also
individual HRTFs through ray-acoustic simulations.
allows, in certain situations, to discard some of the wave phenom-
The paper is organized as follows: After this introduction, we
ena, especially for the higher frequency bands. As a result, two
review in Section 2 the existing approaches for sound simulations
main approaches have evolved for the simulation of sound wave
and compare their advantages and drawbacks. Section 3 follows
propagation: The wave-based and the ray-oriented techniques, with
up on the ray-based approach and develops a model for the prop-
the first one concentrating on the lower and the last one on the mid-
agation of acoustic energy in enclosures. This model studies the
dle and higher frequency ranges. Here Section 2 has a closer look
flow of acoustic energy from sound sources, its local interaction
on both techniques and compares them in terms of efficiency and
with objects and materials, as well as the measurement using a
applicability. Although several improvements have been reported
scene mounted listener. The following Section 4 maps the here
for both techniques, sound simulations are in general performed
developed concepts onto graphics primitives and rendering tech-
offline and are valid only for certain frequency ranges. Due to ad-
niques, and discusses its implementation using modern program-
vances in computational power, as well as in computer graphics
mable graphics hardware. Section 5 presents and discusses results
and acoustics, interactive and dynamic ray-based sound simula-
using examples from room acoustic simulations and personalized
tions are feasible also for complex and more difficult scenes.
HRTF measurements. The closing Section 6 summarizes the work
Accelerated and driven by computer games and the demand
and discusses several ideas for future improvements.
for an even higher visual realism, computer graphics hardware
has evolved tremendously over the last decade and nowadays out-
performs the CPU in terms of computational capacity by several 2. ACOUSTIC SIMULATION TECHNIQUES
magnitudes. As of the easy availability of this processing power,
graphics hardware has been exploited in a number of non-graphics Auralization is defined as the simulation and reproduction of the
calculations, such as solving differential equations, as well as for acoustic properties describing a virtual scene, which has applica-
simulations and numerical analyses [1]. The GPU is, in general, tions in many areas, including architectural design, sound and mu-

DAFX-1
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

sic production and even audio-based computer games [7]. An ac- Although, the simulations using waveguide meshes are very
curate and efficient simulation is thereby still a difficult and com- accurate, there are some drawbacks as well. The two major prob-
putationally extensive task. lems are a direction dependent dispersion error, and a finite mesh
The most often employed approaches are waveguide meshes resolution to model a more complex boundary behavior [8]. Sev-
and raytracing techniques, see also Figure 1. Figure 1(a) displays eral approaches have been discussed to overcome these limitations
here a visualization of the waveguide technique, a more physi- and include higher tesselated meshes, different mesh topologies
cally correct wave-based sound propagation model, based on time- and frequency warping techniques [12, 13]. Additionally, the sam-
domain finite difference meshes. The acoustic energy, eg. pres- pling frequency of the rooms impulse response needs to be over-
sure, is distributed along sampling points using difference equa- sampled, with previous research showing that a typical waveguide
tions. Figure 1(b) shows a visualization of the ray-based approach mesh gives a valid bandwidth only as far as fupdate /4 [8]. There-
that approximates sound waves through particles and acoustic en- fore, this technique is only practical to the very lower frequency
ergy, and were raytracing techniques are used to determine the vir- end. However, recent research has shown that waveguide meshes
tual soundfield. As both techniques have their own advantages and can easily and efficiently be implemented using graphics hardware.
limitations, the wave-oriented techniques are usually employed for Combined with a new sampling lattice, the performance increase
the lower frequency end, while the ray-based techniques are used was measured by a factor of 25, and even more for finer mesh
for the middle and higher frequency parts. resolutions [5].

2.2. Geometric Acoustics

Geometric acoustics is based on optical fundamentals and light
propagation and approximates sound waves through particles mov-
ing along directional rays [14, 15]. These rays are traced through
a virtual scene, starting at the sound source and towards a listen-
ers position, at which the accumulated energy is later evaluated.
As sound waves are now simplified as rays, wave phenomena and
differences in wavelength are usually discarded and ignored. This
(a) Wave-based Approach. method is therefore only applicable to frequencies whose wave-
length are much shorter than the dimensions of the enclosure, or
any object within, refer also to [16, 17].
Several articles have been published over the last years, which
discuss the realtime possibilities of ray-acoustic sound simulations
[18, 19]. The majority of implementations, however, employs
raytracing only to determine specular reflections using ray/beam-
tracing approaches and uses conventional 3D sound APIs for spa-
tialization and sound rendering [14, 15, 4]. As raytracing is a long
known area of research in computer graphics, several improve-
ments and advancements to the original approach have been pro-
(b) Ray-based Approach. posed, and were partially applied to ray-acoustics as well. Savioja
et.al. have designed the DIVA auralization system based on a ray-
acoustics approach, to examine modeling techniques for virtual
Figure 1: Acoustic Simulation Techniques. acoustics, as well as for physically-based auralizations [20, 21].
Some of the more recent geometric acoustic implementations
already utilize computer graphics hardware to increase the simula-
2.1. Wave-based Acoustics tions efficiency. Jedrzejewski uses the GPU for simple 2D geomet-
ric room acoustics using rays and specular reflections [4], while
Wave-based room acoustics is concerned with the numerically eval-
Kapralos and Deines employ a particle-based system to adopt the
uation of the wave equation in order to simulate sound wave prop-
phonon mapping technique towards a phonon tracing approach
agation. Often employed techniques are finite element methods
[22, 23, 24]. Although, this technique allows an accurate mod-
(FEM) and 3D waveguide meshes (time-domain difference mod-
eling of acoustic materials and sound propagation, it only permits
els) [8, 9]. The 1-dimensional waveguide technique is a numer-
static and non-changing environments. Interesting, from the per-
ical solution to the wave equation and was first applied to simu-
spective of a complete GPU-based sound simulation and rendering
late string-based musical instruments [10]. The digital waveguide
approach, is also the work by Gallo and Whalen [3, 2], who em-
mesh is an extension of the 1D technique and constructed by bi-
ploy the GPU as DSP for sound signal filtering and synthesis.
linear delay lines that are arranged in a mesh-like structure [8].
Higher dimensions are built by scattering junctions that are con-
nected to the delay lines and act as spatial and temporal sampling 3. ACOUSTIC ENERGY PROPAGATION
points. The equations that govern the rectilinear waveguide mesh
are based on difference equations derived from the Helmholtz equa- Sound is the propagation of mechanical energy in the form of
tion by discretizing time and space [11]. Depending on the mesh’s pressure variations and can be described by attributes such as fre-
resolution and the internodal sampling distance, the simulations quency, wavelength, speed of propagation etc. Light on the other
can be rather expensive. Due to advances in computing power, re- hand is an electromagnetic radiation, which is described by simi-
altime wave-based room acoustics is feasible for smaller meshes. lar, however, largely different quantities. The propagation of light

DAFX-2
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

fj frangej (Hz) fcenterj (Hz) λcenterj (m) ume element per unit time. The quantity for describing and mea-
suring acoustic energy is radiant power Φ, or flux, and measured
f0 22 – 44 31.5 10.88
in W att or Joule/sec [6]. The intensity is thereby described as
f1 44 – 88 63 5.44
the amount of acoustic energy flowing from/to/through a surface
f2 88 – 177 125 2.74
element per unit time:
f3 177 – 354 250 1.37
f4 354 – 707 500 0.68 dΦ
f5 707 – 1,414 1,000 0.343 I(t) = dt. (2)
dA
f6 1,414 – 2,828 2,000 0.172
The transfer of acoustic energy using a participating media
f7 2,828 – 5,657 4,000 0.086
(air) is characterized by the energy transport theory. The energy
f8 5,657 – 11,314 8,000 0.043
density in the medium of propagation is hereby the sum of the
f9 11,314 – 22,627 16,000 0.021
kinetic and potential energy per unit volume dV and time E(t) =
Ekin (t) + Epot (t) [25]. The kinetic energy density is defined as
Table 1: Frequency Bands fj . the pressure of a sound wave as:

1 M v2 1
energy and its interaction with objects can be measured and de- Ekin (t) = dt = ρ0 v 2 dt, (3)
2 V0 2
scribed by using techniques of radiometry, from which global il- with v being the average velocity of air particles, ρ0 the av-
lumination models used in computer graphics are derived [6]. The erage media density and VM0 its mass per unit volume V0 . The
concepts of radiometry, along its properties and equations, can be potential energy density can be derived from the gas law as:
mapped to the propagation of acoustic energy as well. This as-
R
sumes that the propagation of sound waves can be simplified to pdp 1 p2
a ray-based approach by largely neglecting characteristics such as Epot (t) = 2 dt = dt, (4)
c ρ0 2 c2 ρ 0
wavelength, diffraction and interference. For middle- and higher
frequencies, and depending on the rooms and enclosed objects with p as the pressure of the sound wave and c as the speed
size, this assumption is true to a certain degree. Especially at the of sound in this medium, and therefore defines the total amount of
lower frequency end wave-based effects become such prominent acoustic energy density [25] as:
that they prevail. Therefore, the here discussed model also ad-
dresses these issues and incorporates the wavelength to approxi-
1 p2
mate diffraction and interference effects. The following sections E(t) = Ekin (t) + Epot (t) = (ρ0 v 2 + 2 )dt. (5)
discuss the theories behind, and extend the concepts of radiometry 2 c ρ0
towards a ray/energy-based acoustic propagation model suitable Equation 5 is valid at any position and time within the virtual
for sound wave simulations. auditory environment and serves as basis to describe an acoustic
energy propagation model. In order to quantitatively measure flux
3.1. Emission and Radiation per unit projected surface area and per unit angle, radiance is in-
troduced with:
In order to study and describe the propagation of sound waves us-
ing raytracing techniques, an adequate propagation model that in- d2 Φ
corporates time- and frequency dependencies needs to be defined. L(x, Θ) = , (6)
dωdAcosθ
This can be realized in analogy to the physics of light transporta-
which varies with position x and the ray’s direction Θ. By
tion and global illumination models [6], which now have to be
incorporating the wavelength λj of the frequency bands used (ref.
extended and adopted towards acoustic properties and an acoustic
Table 1), Equation 6 is redefined to:
energy propagation [25].
Whereas the wavelength of the visible spectrum ranges only Z
between 380 nm to 780 nm, the wavelength in acoustics spreads L(x, Θ, fj ) = L(x, Θ, fj )dλ. (7)
As
from 17 mm at 20 kHz up to 17 m at a frequency of 20 Hz. The
frequencies in the audible spectrum are classified and described by The acoustic energy interacting with a surface element can
frequency bands (octaves) according to human psychoacoustics. be further differentiated in incident Ei (incoming) and exitant Ee
In the following sections fj describes a certain frequency band, (outgoing) energy, and is also measured in W att/m2 :
with j being the index number and j+1 the next higher octave. Ta-
ble 1 provides an overview of the different frequency bands, along dΦ
Ei = , Ee = kEi . (8)
their index number, frequency range frangej , center frequency dA
fcenterj and center wavelength λcenterj . The audible spectrum The scalar k is herby defined over [0, 1] and describes the re-
Aspectrum is therefore defined as the sum of these 10 frequency flectivity of the surface with Esurf ace = Ei − Ee and is affected
bands: by the surface material definition. Using a lossless participating
media, the exitant radiance at one point L(x1 → Θ) is exactly
X
9
the same as the incident radiance at another point receiving this
Aspectrum = As = fj . (1) amount of energy L(x2 ← Θ) [6]. Using a density function and
j=0 volume elements, p(x)dV defines the physical number of sound
Similar to light, acoustic energy can be described as the amount particles carrying an acoustic energy quant. If moved in time dt
of pressure variations per unit volume and time, or more accu- across a differential surface area dA, and by using the direction
rately, by the changes in velocity of air particles contained in a vol- ω and speed of propagation c; N = p(x, ω, fj )cdtdAcosθdωdλ

DAFX-3
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

describes the number of particles flowing through this surface ele- Transmission is defined as the energy that passes through an
ment. The radiance per unit volume is accordingly redefined to: object. We redefine this term to describe the frequency-weighted
Z Z amount of energy that passes through an object unaltered and with-
c out refraction. In acoustics, objects smaller than the wavelength of
L(x, Θ, fj ) = p(x, ω, fj )h dλ. (9)
As λ j an incoming sound wave do not interfere, instead the wave simply
An energy/sound source emits acoustic energy that is propa- diffracts around the object and continues unchanged. An according
gated through and by the participating media. The energy radiates frequency dependent modeling of energy transmission can be real-
through an emittance pattern, which can be homogenous in any di- ized using an objects bounding box or sphere that simply transmits
rection, eg. spherically, or direction dependent, such as a cone. As all acoustic energy whose wavelength is equal or above the objects
with light, also acoustic energy attenuates with distance using the size:
familiar inverse square law. Furthermore, atmospheric absorption
occurs, at which certain frequencies are absorbed by the propagat- X
9

ing media. However, this factor is very small and can safely be Letransmitted (x → (π + Θ)) = Eij τfj . (11)
ignored for smaller enclosures, but becomes more prominent with j=0

increasing distances. Here Letransmitted (x → (π + Θ)) describes the amount of

An observer, or listener, can be placed anywhere within the exitant energy per ray for all bands, which simply pass along the
scene to record the acoustic energy present at this location. The direction opposite to the incoming ray, i.e. the ray’s original direc-
listener does not interfere or participate in the energy propagation, tion. The term τfj is used for a finer modeling and a frequency-
but, if required, such as for binaural listening, an additional geom- weighting of the transmission effects.
etry can be placed nearby to simulate head-shadowing effects. The Reflection and diffuse scattering are probably the two most
incoming rays are then weighted and filtered using HRTFs regard- important qualities in acoustic raytracing and can be very well de-
ing the ray’s direction and delay. scribed using bidirectional reflection distribution functions (BRDF)
[6]. A BRDF is defined for a point x as the ratio of the differential
3.2. Local acoustic Energy Exchange radiance reflected in an exitant direction Θe and the differential
The most interesting part in a ray-based acoustic simulation is the irradiance incident through an incoming angle Θi :
interaction and exchange of acoustic energy with objects and sur-
dL(x → Θe )
face elements. Depending on the objects size and the acoustic ma- brdfref lected (x, Θi → Θe ) = . (12)
terial parameters specified, some of the incoming energy might get dE(x ← Θi )
absorbed, reflected, refracted or transmitted, with the total amount The BRDF is frequency dependent, but direction independent,
of energy according to Equation 8 being constant. eg. fr (x, Θi → Θe ) = fr (x, Θe → Θi ) [6, 26]. Diffuse scat-
tering uniformly reflects the incoming acoustic energy in all di-
rections. In acoustics, this behavior is largely influenced by the
surface roughness, which can be used to determine a specular re-
flection coefficient that describes the ratio between specular and
diffuse reflections. Using a complete diffuse scattering, the ra-
diance is independent from the angle of exitance and the BRDF
defined as:
ρdif f use
brdfref lected (x, Θi ↔ Θe ) = , (13)
π
in which the reflectance ρdif f use represents the fraction of
incident energy reflected at the surface. Pure specular reflection
on the other hand diverts all incident energy in only one direction
Figure 2: Local acoustic Energy Exchange. R, which can be simply computed using the law of reflection and
the surface normal N : 2(N (π + Θe ))N − (π + Θe ). A frequency
Figure 2 shows a schematic of the local acoustic energy ex- dependent BRDF for acoustic raytracing can be modeled through:
change. The four effects of absorption, reflection, refraction and
transmission are described in more detail in the remainder of this X
9

section. Every ray that is cast into the scene contains, depending Leref lected (x ← Θi ) = Eij υfj , (14)
on the sound source emittance of course, the energy of all fre- j=0

quency bands. The energy contribution of each ray is evaluated in which υfj is a weighting factor per frequency band fj . The
at the point of intersection with the surface patch using the ray’s majority of materials, however, exhibit a sort of glossy surface, a
length, as well as the surface material properties defined. combination of specular reflection and diffuse scattering.
Some of the incident acoustic energy is thereby usually ab- Refraction occurs at the crossing of two different isotropic me-
sorbed, converted into heat and dissipated back into the system. dia and can be computed similar to the reflection term in Equa-
The absorption is frequency dependent and characterized by a fre- tion 12, except that the outgoing angle Φ of the refracted ray is
quency band coefficient αfj : determined using Snell’s Law: sinΦ = ηη22 . Here η1 and η2 are
the refraction indices of their respective media. A frequency band
X
9
Leabsorbed (x ← Θ) = Eij αfj . (10) weighted refraction can be defined similar to Equation 14 by using
j=0 νfj as weighting coefficient per frequency band.

DAFX-4
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

3.3. Diffraction and Interference Interference describes the superposition of two or more sound
waves and the resulting changes in amplitude. Using a ray-acoustic
Edge diffraction and interference are acoustic phenomena that can sound simulation, interference effects can only be approximated
be modeled accurately using wave-based techniques, but do not fit roughly using the ray’s length and the center wavelength λcenterj
well into the concept of ray-acoustic simulations. However, both of the current frequency band fj . By using an additional scalar as-
are very important and prevail especially in the lower frequency sociated with each ray, also the modeling of phase-preserving and
ranges. Therefore, and in order to obtain a more realistic simu- phase-reversing reflections are possible. The next section focusses
lation, these effects have to be included, or at least approximated. after these theoretical discussions on the implementation of the
Generally, this is done by combining wave-based and ray-based here described acoustic energy propagation model using efficient
approaches and by choosing a certain threshold as boundary fre- computer graphics hardware.
quency. But to a certain degree, these effects can also be approxi-
mated within ray-acoustics.
4. GRAPHICS-BASED RAY ACOUSTIC SIMULATIONS

While the last section discussed the propagation of acoustic energy

and its interaction with objects and materials, this section maps
the there developed concepts onto computer graphics primitives
and rendering equations. The presented framework implements a
ray-based acoustic simulation system that exploits modern com-
puter graphics hardware. The system is designed along current
GPU-based raytracing systems [27, 28, 29], which were extended
towards the acoustic energy propagation model as discussed in the
(a) Scene Rendering as seen from Listener’s Position. last section. The advantages and applicabilities of such an imple-
mentation can be summarized as:
• Efficient ray-based acoustic simulation system that incor-
porates wave phenomena,
• Realtime implementation that exploits graphics hardware,
• Built-in visualization of sound wave propagation,
• An eclectic modeling and design of acoustic materials, with
• Applications for impulse response measurements and gen-
eral room acoustics, as well as to
(b) Combined Depth/Edge Map.
• Approximate individualized HRIRs.
The system takes any 3D polygonal mesh as input, which is
pre-processed into a more efficient accessible structure. It allows
an interactive sound simulation for meshes of up to 15,000 poly-
gons. Using a short pulse as sound signal, room impulse response
(RIR), as well as head-related impulse response (HRIR) measure-
ments are possible. Alternatively, a monaural sound file can be
used as input signal, resulting in a spatialized binaural representa-
tion with the virtual rooms imprint. The sound source/listener po-
(c) Top View with original and diffracted Ray. sitions, as well as the acoustic material definitions can be changed
and adjusted interactively. All sound signal processing, including
HRTF convolution and delay filtering, is realized using fragment
Figure 3: Ray Acoustic Diffraction Simulation.
shaders onboard the graphics hardware.

Sound waves with larger wavelength simply bend around edges, 4.1. Auralization Pipeline
such as if an additional sound source was placed at the diffraction
edge. Diffraction effects are in ray/energy acoustics simply mod- The auralization pipeline employed in our system stretches over
eled through ray-bending, according to the ray’s length and its as- the CPU and GPU systems, but the majority of computations is
sociated frequency band fj . As diffraction is dependent on the carried out in graphics hardware. Figure 4 shows an overview of
objects size and the ray’s wavelength, the amount of energy that is the pipeline, along its partition in CPU and GPU related tasks. As
diffracted is determined individually per frequency band fj . The initialization, 3D scene data, as well as sounds and frequency band
maximum possible diffraction angle was herby determined exper- decomposed HRTFs are loaded into texture memory. The sound
imentally using a wave-based sound propagation system [5]. Fig- data is also decomposed into 10 bands and assigned a position and
ure 3 visualizes the concept of the implemented diffraction sys- emittance pattern within the virtual room. Rays are now cast into
tem. It shows a virtual scene from the listener’s perspective (Fig- the scene starting at the listeners position, and the per frequency
ure 3(a)), the constructed edge map (Figure 3(b)) and the by angle band received acoustic energy is accumulated and stored within so
α diffracted ray from a listener to a sound source (Figure 3(c)). For called cubemaps. This cubemap is later evaluated and the sound
each edge in Figure 3(b), additional rays are cast into the scene for data is filtered and delayed using HRTFs according to their po-
diffraction simulation. sition and the ray’s length. The binaural mixdown is performed

DAFX-5
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

patch. Although, all materials are assigned per vertex, no interpo-

Ray-acoustic Simulation lation of neighboring material attributes is performed yet.
Raytracing and Object Interaction, Diffraction
The raycasting and acoustic energy accumulation is carried out
using so called cubemaps. One cubemap is hereby centered around
Scene HRTF Texture
Evaluation 10 Bands
the observers position and a ray is cast into the scene per cubemap
texel. Figure 5 shows a visualization of this cubemap raycasting
Filtering and Sound Texture approach. Each ray cast is traced through the virtual scene and
Synthesis 10 Bands
its acoustic energy accumulated and stored per frequency band.
Mixdown Frequency Decomposition
At points of ray/object intersection, the local surface acoustic en-
L/R Channel 10 Bands ergy exchange is evaluated according to Section 3.2. Newly gener-
GPU ated rays from refraction, transmission and/or reflection are further
CPU traced, until their possible energy contribution falls below a certain
Sound Data HRTF Data threshold . The cubemap not only stores all incoming acoustic
3D Scene Data energy per frequency band, but also the ray’s direction and length.
Buffer Playback User Interaction Uniform Grid
OpenAL Listener / Sound Position Acoustic Material This information is later used for the final binaural sound signal
synthesis.
Figure 4: Auralization Pipeline.

using a two-channel floating point texture, which is streamed back

to the CPU and fills a native OpenAL stereo buffer for sound play-
back.

4.1.1. Uniform Grid Structure

In a pre-processing step, the 3D scene is converted into a uniform
grid structure that subdivides 3D space and groups neighboring
triangles together in a voxel-based topology. These voxels are of
uniform size and axis aligned. This space subdivision is necessary
in order to efficiently determine ray/object intersections, as now
only the triangles grouped in one voxel element have to be tested
[27, 28, 29]. Care has to be taken in defining the voxel’s size, Figure 5: Ray Energy Accumulation Buffer.
as with very detailed objects the number of polygons can easily
exceed the number of possible shader instructions.
4.2.1. Diffraction Simulation
4.1.2. Frequency Decomposition and Synthesis
The ray acoustic simulation also incorporate diffraction effects on
A frequency-based acoustic raytracing has many advantages, as edges and object borders. To find possible diffraction locations,
now some of the wave-based propagation effects can be approx- a depth/edge map is employed, which highlights these edges, see
imated, as well as it allows a more realistic frequency-dependent also Figure 3(b). These maps are created by using the scenes depth
definition of acoustic materials. Currently we employ 10 frequency buffer and an image-based edge detection algorithm. If a ray is cast
bands, grouped into octaves as known from psychoacoustics, see close to a diffraction edge, the ray is bend according to the diffrac-
Table 1. For the frequency decomposition of sound data and HRTFs, tion of sound waves [16, 17]. Here the ray’s energy is attenuated,
we employ a time-based convolution using windowed sinc filters, depending on the angle and the ray’s wavelength. Another wave-
with their cutoff frequencies specified as the bands respective bor- based phenomena is interference, which can be roughly approx-
der frequencies. These 10 bands are loaded as floating point tex- imated by using the ray’s length and the frequency bands center
tures into graphics hardware. To remain data precision, we cur- wavelength λcenterj , see Table 1. Although, this is a very sim-
rently employ 3 RGBA textures to hold the sound data, although, ple approximation, it would also allow the modeling of phase-
using data compression, two should be sufficient for 16 bit sound reversing and -preserving boundary reflections, as well as to use
data. Ray/object interactions are evaluated per frequency band and this information for interference effects of the same and/or differ-
the contributions from each ray are accumulated and also stored ent frequency bands.
individually. The final auralization is a binaural sound signal that
is generated by filtering the original sound texture using HRTFs 4.3. Implementation
according to the simulations result.
Today’s graphics hardware, and especially the new generation with
4.2. Acoustic Raytracing and Diffraction Simulation its unified shader architecture, can be seen as powerful parallel
processing machines, which can very efficiently execute small pro-
The authoring of 3D scenes can be conveniently performed using grams - so called shaders - in parallel. Shaders are freely program-
3D Studio MAX, where a custom-built plugin is used to assign mable using high level shading languages such as GLSL and Cg.
acoustic material definitions to each object. This acoustic material As graphics applications typically require the processing of huge
defines the wavelength specific energy exchanges for each surface amounts of data, graphics hardware has been optimized to support

DAFX-6
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

this with a highly parallel design. Combined with a fast and op-
timized memory architecture for accessing and storing the data,
this makes this hardware very interesting for any computationally
intensive and parallelizable task.
All convolutions and sound synthesis are carried out using
fragment shaders on graphics hardware, with a single shader for
each task. The data, eg. sounds, geometry and material definitions
are stored within textures and accessed during the rendering task
from within the shaders. The results of the simulation are again
stored as textures, from which they are read back to the CPU for (a) Church. (b) Living Room.
sound playback.
1 1

Intensity

Intensity
5. RESULTS AND DISCUSSION

This section discusses some results of the ray-acoustics simula-

tion system. The implementation is based on nvidia type graphics
hardware and uses Cg as shading language. The current experi-
ments were performed with three different graphics hardware gen-
Time in ms 100 Time in ms 100
erations, showing that only the newest one (GeForce8800GTX)
was also able to additionally perform a realtime auralization of the (c) Echogram Church. (d) Echogram Living Room.
results besides the sound simulation. Frame rates of up to 25 f ps
could be achieved using a detailed model of a living room ( 1,500 Figure 7: Room Acoustic Example Scenarios.
polygons) including a binaural auralization of the scene, ref. Fig-
ure 7(b).
echograms show the early reflections, as well as late reverberation
and diffraction effects.

5.2. Example 2: Personalized HRTF

The second example shows an HRIR simulation of the horizontal
plane using our ray-acoustics approach. The simulation was per-
formed using 72 sound sources, each 1.2 m apart from the head
at a 5 degree interval. Although, the simulation does not exhibit
all effects of a regular measured HRIR, it shows the most promi-
nent features. The simulation was performed using a 3D model
of the KEMAR mannequin. Figure 8 shows two different simula-
(a) Transmission and Refraction. (b) Diffraction around Pillars.
tion results, along the original 3D model used. Here Figure 8(a)
displays an HRIR simulation of the system from [30], while Fig-
Figure 6: Evaluation of Sound Propagation Effects. ure 8(b) show the results of the here presented ray-acoustics sys-
tem. Thereby roughly 18 million rays were traced per sound source,
Figure 6 shows two visualizations of sound propagation ef- resulting in a simulation time of 22 seconds per sound source. Al-
fects. Here Figure 6(a) displays the transmission and refraction though the most important features are clearly present, several ef-
parts of the simulation, whereas Figure 6(b) shows diffraction ef- fects are still missing. This is partially due to the fact that we only
fects of several pillars. In both cases the sound source is hidden consider one diffraction per ray. Also, a more detailed fine tuning
and the simulation results are visible in the unfolded cubemaps of parameters along the material definitions for the head, torso and
below. Both cubemaps show a red/braun shifting of the color, de- ear will yield better results. The goal is to combine an individu-
noting a stronger transmission/diffraction in the lower frequencies. alized HRTF simulation with room acoustics, to yield a realtime
personalized binaural room simulation. Better results with geo-
5.1. Example 1: Room Acoustics metric models have been achieved by [31], but, however, also with
a much longer simulation time.
The first example shows two different rooms along their echograms.
Figure 7(a) displays thereby a small church, while Figure 7(b)
shows an average living room. The echogram of the church, ref. 6. CONCLUSIONS AND FUTURE WORK
Figure 7(c) shows strong and late echoes, while the echogram in
We have presented a realtime graphics-based implementation of a
Figure 7(d) shows that nearly all acoustic energy, except the direct
ray acoustic simulation system that is based on an acoustic en-
line, was absorbed by walls and furniture. Both echograms clearly
ergy propagation model. This underlying model is founded on
visualize the rooms acoustic properties. Each room has been mod-
sound propagation, as well as global illumination models, and the
eled using 3D Studio MAX, in which for each surface a differ-
ray/energy approach used therefore valid and its implementation
ent materials has been specified. The properties for the acoustic
using graphics hardware and techniques viable. The current re-
material definitions were taken from the CARA database1 . The
sults clearly show the possibilities of this system and motivate a
1 http://www.cara.de further research in this area.

DAFX-7
Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx-07), Bordeaux, France, September 10-15, 2007

[10] Julius O. Smith, “Physical modelling using digital Waveguides,”

Computer Music Journal, vol. 16, no. 4, pp. 75–87, 1992.
[11] Lauri Savioja and Tapio Lokki, “Digital Waveguide Mesh for Room
Acoustic Modelling,” in ACM SIGGRAPH Campfire: Acoustic Ren-
dering for Virtual Environments, Utah, USA, 2001.
[12] Mark J. Beeson and Damian T. Murphy, “Roomweaver: A digital
Waveguide Mesh based Room Acoustics Research Tool,” in 7th Int.
Conference on Digital Audio Effects, Italy, 2004, pp. 268–273.
[13] F. Fontana and D. Rocchesso, “Signal-Theoretic Characterization of
Waveguide Mesh Geometries for Models of Two-Dimensional Wave
Propagation in Elastic Media,” IEEE Transactions on Speech and
(a) HRIR 1. (b) HRIR 2. (c) KEMAR Model. Audio Processing, vol. 9, no. 2, pp. 152–161, Februar 2001.
[14] Wolfgang Mueller and Frank Ullmann, “A Scalable System for 3D
Figure 8: HRIR Simulation and 3D Model. Audio Ray Tracing,” in Proceedings of the IEEE Int. Conference on
Multimedia Computing and Systems, 1999, pp. 819–823.
[15] Thomas Neumann, “Eine Geometrie-Engine zur Berechnung von
The current implementation already exhibits good and very 3D-Sound mit Raumakustik-Effekten in Echtzeit,” M.S. thesis, Tch-
promising results, yet some ideas are left for future improvements. nische Universität Braunschweig, Institut für Compuergraphik, 2004.
One extension would be an enhanced space partitioning structure, [16] F. Alton Everest, The Master Handbook of Acoustics, vol. 4th edition,
such as kD-Trees that allow a non-uniform subdivision of 3D space. McGraw-Hill/TAB Electronics, 2001.
Future work also includes more and finer partitioned frequency [17] Heinrich Kuttruff, Room Acoustics, Spon Press, London, 2000.
bands for a more accurate studying and modeling of wave-based [18] Thomas Funkhouser, Nicolas Tsingos, Ingrid Carlbom, Gary Elko,
propagation effects. Another beneficial extension would be a higher Mohan Sondhi, and James West, “Modeling Sound Reflection and
incorporation of radiosity techniques, although one has to be care- Diffraction in Architectural Environments with Beam Tracing,” Fo-
rum Acusticum, 2002.
ful to not impede here with realtime simulations and dynamic en-
vironments. Additionally, ambisonics and their implementation [19] Emmanuel Tsingos, Nicolas Gallo and George Drettakis, “Perceptual
Audio Rendering of Complex Virtual Environments,” in Proceedings
using spherical harmonics in realtime computer graphics might be
of ACM SIGGRAPH, 2004.
an interesting path to explore.
[20] Lauri Savioja, Tapio Lokki, and Jyri Huopaniemi, “Auralization ap-
plying the Parametric Room Acoustic Modelling Technique - The
7. ACKNOWLEDGEMENTS DIVA Auralization System,” in Proceedings of ICAD, Japan, 2002.
[21] Lauri Savioja, Modelling Techniques for Virtual Acoustics, Ph.D.
The authors would like to thank Yuvi Kahana, Brian FG Katz, as thesis, Helsinki University of Technology, Finnland, 2000.
well as Richard O. Duda for providing a polygonal mesh of the [22] B. Kapralos, M. Jenkin, and E. Milios, “Sonel Mapping: Acoustic
KEMAR head for the HRIR simulations. Modeling utilizing an acoustic Version of Photon Mapping,” in IEEE
International Workshop on Haptic Audio Visual Environments and
8. REFERENCES their Applications, 2004.
[23] Martin Bertram, Eduard Deines, Jan Mohring, Jevgenij Jegorovs, and
[1] John D. Owens, David Luebke, Naga Govindaraju, Mark Harris, Jens Hans Hagen, “Phonon Tracing for Auralization and Visualization of
Krüger, Aaron E. Lefohn, and Timothy J. Purcell, “A survey of Sound,” in IEEE Visualization, Minneapolis, USA, 2005.
general-purpose computation on graphics hardware,” in Eurograph- [24] Eduard Deines, Martin Bertram, Jan Mohring, Jevgenij Jegorovs,
ics 2005, State of the Art Reports, Aug. 2005, pp. 21–51. Frank Michel, Hans Hagen, and Gregory M. Nielson, “Compara-
[2] Sean Whalen, “Audio and the Graphics Processing Unit,” Tech. Rep., tive Visualization for Wave-based and Geometric Acoustics,” IEEE
2005, http://www.node99.org/projects/gpuaudio/. Transactions on Visualization and Computer Graphics, vol. 12, no.
[3] Emmanuel Gallo and Nicolas Tsingos, “Efficient 3D Audio Process- 5, September/October 2006.
ing with the GPU,” in GP2, ACM Workshop on General Purpose [25] Leo L. Beranek, Acoustics, Amer Inst of Physics, 1986.
Computing on Graphics Processors, 2004. [26] Zhiyun Li, Ramani Duraiswami, and Nail A. Gumerov, “Capture
[4] Marcin Jedrzejewski, “Computation of Room Acoustics Using Pro- and Recreation of higher Order 3D Sound Fields via Reciprocity,” in
grammable Video Hardware,” in International Conference on Com- ICAD, 200f.
puter Vision and Graphics, 2004. [27] Timothy J. Purcell, Ian Buck, William R. Mark, and Pat Hanrahan,
[5] Niklas Röber, Martin Spindler, and Maic Masuch, “Waveguide-based “Ray Tracing on Programmable Graphics Hardware,” in Proceedings
Room Acoustics through Graphics Hardware,” in ICMC, 2006. of ACM SIGGRAPH, 2002, pp. 703–712.
[6] Philip Dutre, Philippe Bekaert, and Kavita Bala, Advanced Global [28] Timothy J. Purcell, Ray Tracing on a Stream Processor, Ph.D. thesis,
Illumination, AK Peters Ltd., 2003. Stanford University, March 2004.
[7] Niklas Röber and Maic Masuch, “Leaving the Screen: New Perspec- [29] Gabriel Moreno-Fortuny, “Distributed Stream Processing Ray
tives in Audio-only Gaming,” in Proceedings of 11th ICAD, 2005, Tracer,” Tech. Rep., University of Waterloo, http://www.cgl.
Limerick, Ireland. uwaterloo.ca/~gmoreno/streamray.html 2004.
[30] Niklas Röber, Sven Andres, and Maic Masuch, “HRTF Simulations
[8] S. VanDuyne and J.O. Smith, “Physical Modelling of the 2-D dig-
through acoustic Raytracing,” Tech. Rep., Fakultät für Informatik,
ital Waveguide Mesh,” in Int. Computer Music Conference, Tokyo,
Otto-von-Guericke Universität Magdeburg, 2006.
Japan, 1993, pp. 40–47.
[31] V.R. Algazi, R.O. Duda, R. Duraiswami, N.A. Gumerov, and Z. Tang,
[9] Y. Kahana, P.A. Nelson, M. Petyt, and S. Choi, “Numerical Mod- “Approximating the Head-related Transfer Function using simple
elling of the Transfer Functions of a Dummy-Head and of the exter- Geometric Models of the Head and Torso,” Journal of the Acoustical
nal Ear,” in Audio Engineering Society, 16th International Confer- Society of America, , no. 5, pp. 2053–2064, November 2002.
ence, Rovaneimi, 1999, pp. 330–345.

DAFX-8