article

Integrating Sounds and Motions in Virtual Environments

Authors:

Jong Won LeeAuthors Info & Claims

Presence: Teleoperators and Virtual Environments, Volume 7, Issue 1

Pages 67 - 77

https://doi.org/10.1162/105474698565532

Published: 01 February 1998 Publication History

Abstract

Sounds are often the result of motions of virtual objects in a virtual environment. Therefore, sounds and the motions that caused them should be treated in an integrated way. When sounds and motions do not have the proper correspondence, the resultant confusion can lessen the effects of each. In this paper, we present an integrated system for modeling, synchronizing, and rendering sounds for virtual environments. The key idea of the system is the use of a functional representation of sounds, called timbre trees. This representation is used to model sounds that are parameterizable. These parameters can then be mapped to the parameters associated with the motions of objects in the environment. This mapping allows the correspondence of motions and sounds in the environment. Representing arbitrary sounds using timbre trees is a difficult process that we do not address in this paper. We describe approaches for creating some timbre trees including the use of genetic algorithms. Rendering the sounds in an aural environment is achieved by attaching special environmental nodes that represent the attenuation and delay as well as the listener effects to the timbre trees. These trees are then evaluated to generate the sounds. The system that we describe runs parallel in real time on an eight-processor SGI Onyx. We see the main contribution of the present system as a conceptual framework on which to consider the sound and motion in an integrated virtual environment.

References

[1]

Cook, R. (1984). Shade trees. Proceedings of SIGGRAPH 84, 18(3), 195-206.

[2]

Dannenberg, R., Fraley, C., & Velikonj, P. (1991). Fugue: A functional language for sound synthesis. IEEE Computer, 24(7), 36-42.

Digital Library

[3]

Fouad, H., & Hahn, J. (1996). A framework for integrating audio in virtual environments. Proceedings of the SPIE Symposium on Electronic Imaging: Science & Technology, 2653, 285-291.

[4]

Funkhouser, T., & Sequin, C. (1993). Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments. Proceedings of SIGGRAPH 93, 247-254.

[5]

Gaver, W. (1993). Synthesizing auditory icons. Proceedings of INTERCHI, 228-235.

[6]

Gritz, L., & Hahn, J. (1995). Genetic programming for articulated figure motion. Journal of Visualization and Computer Animation, 6(3), 129-142.

[7]

Hahn, J. (1988). Realistic animation of rigid bodies. Proceedings of SIGGRAPH '88, 22(3), 299-308.

[8]

Hahn, J., Gritz, L., Darken, R., Geigel, J., & Lee, J. (1993). An integrated virtual environment system. Presence: Teleoperators and Virtual Environments, 2(4), 353-360.

Digital Library

[9]

Hahn, J., Geigel, J., Lee, J., Mishra, S., Gritz, L., & Takala, T. (1995). An integrated approach to motion and sound. Journal of Visualization and Computer Animation, 6(2), 109- 123.

[10]

Koza, J. (1992). Genetic programming. Cambridge, MA: MIT Press.

[11]

Mathews, M. (1969). The technology of computer music. Cambridge, MA: MIT Press.

[12]

Mishra, S., & Hahn, J. (1995). Mapping motion to sound and music in computer animation and VE. Proceedings of Pacific Graphics, 83-98.

[13]

Moore, F. (1990). Elements of computer music. Englewood Cliffs, NJ: Prentice Hall.

[14]

Perlin, K. (1985). An image synthesizer. Proceedings of SIGGRAPH '85, 19(3), 287-296.

[15]

Pope, S., & Fehlen, L. (1993). The use of 3-D audio in a synthetic environment: An aural renderer for a distributed virtual reality system. Proceedings of the IEEE Virtual Reality Annual International Symposium, 176-182.

[16]

Scaletti, C. (1991). The Kyma/Platypus computer music workstation. In S. Pope (Ed.). The well-tempered object: Musical applications of object oriented software technology. Cambridge, MA: MIT Press.

[17]

Silicon Graphics, Inc. (1991). Parallel programming on Silicon Graphics computer systems. Version 1.0, Documentation number 007-0770-010.

[18]

Sims, K. (1991). Artificial evolution for computer graphics. Proceedings of SIGGRAPH '91, 25(3), 319-328.

[19]

Takala, T., & Hahn, J. (1992). Sound rendering. Proceedings of SIGGRAPH '92, 26(2), 211-220.

[20]

Takala, T., Hahn, J., Gritz, L., Geigel, J., and Lee, J. (1993). Using physically-based models and genetic algorithms for functional composition of sound signals, synchronized to animated motion. International Computer Music Conference (ICMC), 180-184.

[21]

Thalmann, N., & Thalmann, D. (1990). Synthetic actors in computer generated 3D films. Tokyo: Springer-Verlag.

[22]

Vercoe, B. (1986). Csound: A manual for the audio processing system and supporting programs. Cambridge, MA: MIT Media Lab.

[23]

Watt, A., & Watt, M. (1992). Advanced animation and rendering techniques. Reading, MA: Addison-Wesley.

[24]

Wenzel, E. (1992). Localization in Virtual Acoustic Displays. Presence: Teleoperators and Virtual Environments, 1, 80- 107.

Digital Library

Cited By

Su FJoslin C(2019)Toward Generating Realistic Sounds for Soft BodiesProceedings of the 14th International Audio Mostly Conference: A Journey in Sound10.1145/3356590.3356620(199-206)Online publication date: 18-Sep-2019
https://dl.acm.org/doi/10.1145/3356590.3356620
Bisby HCooper AJames SRobertson ANg KNg KBowen JMcDaid S(2014)A Tactile Audio Visual Instrument using Sound Source LocalisationProceedings of the EVA London 2014 on Electronic Visualisation and the Arts10.14236/ewic/eva2014.24(200-206)Online publication date: 8-Jul-2014
https://dl.acm.org/doi/10.14236/ewic/eva2014.24
Mcgregor I(2014)Comparing designers' and listeners' experiencesAI & Society10.1007/s00146-013-0489-429:4(473-483)Online publication date: 1-Nov-2014
https://dl.acm.org/doi/10.1007/s00146-013-0489-4
Show More Cited By

Index Terms

Integrating Sounds and Motions in Virtual Environments
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing

Index terms have been assigned to the content through auto-classification.

Recommendations

The effects of spatialized sounds on the sense of presence in auditory virtual environments: A psychological and physiological study

Although many studies have indicated that spatialized sounds increase the subjective sense of presence in virtual environments, few studies have examined the effects of sounds objectively. In this study, we examined whether three-dimensional reproduced ...
Using wavelets to synthesize stochastic-based sounds for immersive virtual environments

Stochastic, or nonpitched, sounds fill our real-world environment. Humans almost continuously hear stochastic sounds, such as wind, rain, motor sounds, and different types of impact sounds. Because of their prevalence in real-world environments, it is ...
Evaluating Environmental Sounds from a Presence Perspective for Virtual Reality Applications

We propose a methodology to design and evaluate environmental sounds for virtual environments. We propose to combine physically modeled sound events with recorded soundscapes. Physical models are used to provide feedback to users' actions, while ...

Comments

Information & Contributors

Information

Published In

cover image Presence: Teleoperators and Virtual Environments

Presence: Teleoperators and Virtual Environments Volume 7, Issue 1

February 1998

100 pages

ISSN:1054-7460

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 February 1998

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Su FJoslin C(2019)Toward Generating Realistic Sounds for Soft BodiesProceedings of the 14th International Audio Mostly Conference: A Journey in Sound10.1145/3356590.3356620(199-206)Online publication date: 18-Sep-2019
https://dl.acm.org/doi/10.1145/3356590.3356620
Bisby HCooper AJames SRobertson ANg KNg KBowen JMcDaid S(2014)A Tactile Audio Visual Instrument using Sound Source LocalisationProceedings of the EVA London 2014 on Electronic Visualisation and the Arts10.14236/ewic/eva2014.24(200-206)Online publication date: 8-Jul-2014
https://dl.acm.org/doi/10.14236/ewic/eva2014.24
Mcgregor I(2014)Comparing designers' and listeners' experiencesAI & Society10.1007/s00146-013-0489-429:4(473-483)Online publication date: 1-Nov-2014
https://dl.acm.org/doi/10.1007/s00146-013-0489-4
Zheng CJames DDeRose T(2010)Rigid-body fracture sound with precomputed soundbanksACM SIGGRAPH 2010 papers10.1145/1833349.1778806(1-13)Online publication date: 26-Jul-2010
https://dl.acm.org/doi/10.1145/1833349.1778806
Zheng CJames D(2010)Rigid-body fracture sound with precomputed soundbanksACM Transactions on Graphics10.1145/1778765.177880629:4(1-13)Online publication date: 26-Jul-2010
https://dl.acm.org/doi/10.1145/1778765.1778806
Zhang YFernando TXiao HTravis A(2006)Evaluation of auditory and visual feedback on task performance in a virtual assembly environmentPresence: Teleoperators and Virtual Environments10.1162/pres.15.6.61315:6(613-626)Online publication date: 1-Dec-2006
https://dl.acm.org/doi/10.1162/pres.15.6.613
Sodnik JTomazic SGrasset RDuenser ABillinghurst MRobertson T(2006)Spatial sound localization in an augmented reality environmentProceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments10.1145/1228175.1228197(111-118)Online publication date: 20-Nov-2006
https://dl.acm.org/doi/10.1145/1228175.1228197
Avanzini FCrosato P(2006)Haptic-auditory rendering and perception of contact stiffnessProceedings of the First international conference on Haptic and Audio Interaction Design10.1007/11821731_3(24-35)Online publication date: 31-Aug-2006
https://dl.acm.org/doi/10.1007/11821731_3
Zhang YSotudeh RFernando TJüttler B(2005)The use of visual and auditory feedback for assembly task performance in a virtual environmentProceedings of the 21st Spring Conference on Computer Graphics10.1145/1090122.1090133(59-66)Online publication date: 12-May-2005
https://dl.acm.org/doi/10.1145/1090122.1090133
van den Doel KKnott DPai D(2004)Interactive simulation of complex audiovisual scenesPresence: Teleoperators and Virtual Environments10.5555/1008550.100855813:1(99-111)Online publication date: 1-Feb-2004
https://dl.acm.org/doi/10.5555/1008550.1008558
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents