FreesoundVR: soundscape composition in virtual reality using online sound repositories
Abstract
The intersection between sound and music computing and Virtual Reality (VR) has grown significantly over the past decades, amounting to an established area of research today. However, still scarce research has been conducted on the development of specific tools for sound design and composition. In this paper, we investigate a new way of exploring online sound repositories to retrieve sounds to be used in soundscape composition, which leverages the VR medium. Specifically, we created a VR system that allows users to search, download, and explore Freesound content in an immersive manner, as well as to use it for soundscape composition practices via a virtual digital audio workstation (DAW). The tags associated to a sound in the repository were converted into virtual objects and environments, which the user could navigate while listening to the sound. We conducted a user study with 16 composers where the developed system was compared against a conventional counterpart comprising the Freesound web version and the Audacity DAW. Overall, quantitative and qualitative results did not indicate a clear and generalized preference for a system over the other. The usability of the two systems along with their offered creativity support, cognitive workload and emotional impact were deemed to be at a comparable level. Nevertheless, the full potential of VR in creating novel compositional experiences also clearly emerged. Our study shows that VR is an effective medium to support users’ creativity during the process of exploring and selecting sounds from an online repository as well as for composing a soundscape.
References
[1]
Akkermans V, Font F, Funollet J, de Jong B, Roma G, Togias S, Serra X (2011) Freesound 2: an improved platform for sharing audio clips. Proceedings of the international society for music information retrieval conference
[2]
Barri T (2009) Frameworks 3D: composition in the third dimension. Proceedings of the international conference on new interfaces for musical expression, pp. 264–265
[3]
Bradley MM and Lang PJ Measuring emotion: the self-assessment manikin and the semantic differential J Behav Ther Exper Psych 1994 25 1 49-59
[4]
Braun V and Clarke V Using thematic analysis in psychology Qualitat Res Psychol 2006 3 2 77-101
[5]
Brooke J et al. Sus-a quick and dirty usability scale Usabil Evaluat Ind 1996 189 194 4-7
[6]
Buckley Z, Carlson K (2019) Towards a framework for composition design for music-led virtual reality experiences. IEEE conference on virtual reality and 3d user interfaces pp. 1497–1499
[7]
Carey B (2016) Spectrascore vr: Networkable virtual reality software tools for real-time composition and performance. International conference on new interfaces for musical expression
[8]
Chapman O The icebreaker: Soundscape works as everyday sound art Organised Sound 2009 14 1 83-88
[9]
Cherry E and Latulipe C Quantifying the creativity support of digital tools through the creativity support index ACM Trans Comp-Human Interact 2014 21 4 21
[10]
Ciciliani M Virtual 3D environments as composition and performance spaces J New Music Res 2020 49 1 104-113
[11]
Costa W, Ananias L, Barbosa I, Barbosa B, De’Carli A, Barioni R et al. (2019) Songverse: a music-loop authoring tool based on virtual reality. IEEE symposium on virtual and augmented reality (pp. 216–222)
[12]
Drever JL Soundscape composition: the convergence of ethnography and acousmatic music Organ Sound 2002 7 1 21-27
[13]
d’Escriván J Sound art (?) on/in film Organ Sound 2009 14 1 65-73
[14]
Eckel G Immersive audio-augmented environments: the listen project Proceed Fifth Int Conf Infor Visual 2001 128 571-573
[15]
Favory X, Fonseca E, Font F, Serra X (2018) Facilitating the manual annotation of sounds when using large taxonomies. Proceedings of the 23rd conference of open innovations association fruct, pp. 60–64
[16]
Fonseca E, Pons Puig J, Favory X, Font Corbera F, Bogdanov D, Ferraro A et al. (2017) Freesound datasets: a platform for the creation of open audio datasets. Proceedings of the international society for music information retrieval conference, pp. 486–493
[17]
Font F (2021) SOURCE: a Freesound Community Music Sampler. Proceedings of audio mostly conference, pp. 182–187
[18]
Font F, Brookes T, Fazekas G, Guerber M, La Burthe A, Plans D et al. (2016) Audio commons: bringing creative commons audio content to the creative industries. Audio engineering society conference: 61st international conference: audio for games
[19]
Font F, Roma G, Serra X (2013) Freesound technical demo. Proceedings of the ACM international conference on Multimedia, pp. 411–412
[20]
Freeman J, Disalvo C, Nitsche M, and Garrett S Soundscape composition and field recording as a platform for collaborative creativity Organ Sound 2011 16 3 272-281
[21]
Graham R, Cluett S (2016) The soundfield as sound object: Virtual reality environments as a three-dimensional canvas for music composition. Audio engineering society conference: 2016 AES international conference on audio for virtual and augmented reality
[22]
Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. Proceedings of the human factors and ergonomics society annual meeting, Vol. 50, pp. 904–908
[23]
Huang C, Lin Y, Zeng X, Newman M, O’Modhrain S (2015) Olegoru: a soundscape composition tool to enhance imaginative storytelling with tangible objects. Proceedings of the ninth international conference on tangible, embedded, and embodied interaction, pp. 709–714
[24]
Koutsomichalis, M. (2013). On soundscapes, phonography and environmental sound art. Journal of sonic studies, 4(1).
[25]
Leonard M, Strachan R (2014) More than background: Ambience and sound-design in contemporary art documentary film. Music and sound in documentary film, pp. 180–193. Routledge
[26]
Masu R, Bala P, Ahmad M, Do Nascimento Correia N, Nisi V, Nunes N, Romão T (2020) Vr open scores: scores as inspiration for vr scenarios. Proceedings of the international conference on new interfaces for musical expression, pp. 109–114
[27]
Moore A, Howell M, Stiles A, Herrera N, McMahan R (2015) Wedge: A musical interface for building and playing composition-appropriate immersive environments. IEEE symposium on 3d user interfaces, pp. 205–206
[28]
Polfreman R (2009) Frameworks 3D: composition in the third dimension. Proceedings of the international conference on new interfaces for musical expression, pp. 226–229
[29]
Schafer RM (1977) The tuning of the world. Random House Inc
[30]
Schirosa M, Janer J, Kersten S, Roma G (2010) A system for soundscape generation, composition and streaming. Xvii cim-colloquium of musical informatics
[31]
Skach S, Xambó A, Turchet L, Stolfi A, Stewart R, Barthet M (2018) Embodied interactions with e-textiles and the internet of sounds for performing arts. Proceedings of the international conference on tangible, embedded, and embodied interaction, pp. 80–87. ACM
[32]
Thorogood M, Pasquier P (2013) Computationally created soundscapes with audio metaphor. International conference on computational creativity, pp. 1–7
[33]
Truax B Electroacoustic music and the soundscape: the inner and outer world Comp Contemp Music Thought 1992 1 374-398
[34]
Truax B Soundscape, acoustic communication and environmental sound composition Contemp Music Rev 1996 15 1–2 49-65
[35]
Truax B Soundscape composition as global music: electroacoustic music as soundscape Organ Sound 2008 13 2 103-109
[36]
Turchet L, Barthet M (2018) Jamming with a smart mandolin and Freesound-based accompaniment. IEEE Conference of open innovations association (FRUCT), pp. 375–381
[37]
Turchet L, Fazekas G, Lagrange M, Shokri Ghadikolaei H, and Fischione C The internet of audio things: state-of-the-art, vision and challenges IEEE Intern Thing J 2020 7 10 10233-10249
[38]
Turchet L, Hamilton R, and Çamci A Music in extended realities IEEE Access 2021 9 15810-15832
[39]
Turchet L and Serafin S Investigating the amplitude of interactive footstep sounds and soundscape reproduction Appl Acoust 2013 74 4 566-574
[40]
Turchet L, Zanetti A (2020) Voice-based interface for accessible soundscape composition: composing soundscapes by vocally querying online sounds repositories. Proceedings of audio mostly conference, pp. 160–167
[41]
Turner P, McGregor I, Turner S, Carroll F (2003) Evaluating soundscapes as a means of creating a sense of place. Proceedings of international conference on auditory display, pp. 148–151
[42]
Valle A, Armao P, Casu M, Koutsomichalis M (2014) Soda: A sound design accelerator for the automatic generation of soundscapes from an ontologically annotated sound library. International computer music conference
[43]
Valle A, Lombardo V, Schirosa M (2009) Simulating the soundscape through an analysis/resynthesis methodology. Auditory display, pp. 330–357. Springer
[44]
Verron C, Aramaki M, Kronland-Martinet R, and Pallone G A 3-d immersive synthesizer for environmental sounds IEEE Trans Audio, Speech and Lang Process 2009 18 6 1550-1561
[45]
Westerkamp H Linking soundscape composition and acoustic ecology Organ Sound 2002 7 1 51-56
[46]
Wrightson K An introduction to acoustic ecology Soundscape: The J Acoust Ecol 2000 1 1 10-13
[47]
Zappi V, Mazzanti D, Brogni A, Caldwell D (2012) Concatenative synthesis unit navigation and dynamic rearrangement in vrGrains. Proceedings of the sound & music computing conference, pp. 403–408
Recommendations
Soundscape music: sub-genre of new-age music
AMTA'10: Proceedings of the 11th WSEAS international conference on Acoustics & music: theory & applicationsSince the concept of Soundscape brought forth by Canadian composer R. Murray Schafer in the 1960s, researchers and musicians in different realms started to study the new term. In most instances, researchers prefer to pay attention to Soundscape effects ...
The influence of mood induction by music or a soundscape on presence and emotions in a virtual reality park scenario
AM '20: Proceedings of the 15th International Audio Mostly ConferenceMusic and background sound are often used in virtual realities for creating an emotional atmosphere. The present study investigates how music or an ambient soundscape influence presence, the feeling of "being there", as well as positive and negative ...
Adaptive walk on fitness soundscape
ECAL'09: Proceedings of the 10th European conference on Advances in artificial life: Darwin meets von Neumann - Volume Part IIWe propose a new IEC for musical works based on an adaptive walk on a fitness landscape of sounds. In this system, there is a virtual plane that represents the genetic space of possible musical works called fitness soundscape. The user stands on the ...
Comments
Information & Contributors
Information
Published In
© The Author(s) 2022.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 30 September 2022
Accepted: 19 September 2022
Received: 11 February 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
- Università degli Studi di Trento
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Dec 2024