short-paper

Lundheim: Exploring Affective Audio Techniques in an Action-Adventure Video Game

Authors:

Tyler Howard McIntosh,

Jonathan Weinel,

Stuart CunninghamAuthors Info & Claims

AM '22: Proceedings of the 17th International Audio Mostly Conference

Pages 155 - 158

https://doi.org/10.1145/3561212.3561234

Published: 10 October 2022 Publication History

Abstract

This paper discusses Lundheim, a video game prototype made in Unity that incorporates interactive mechanisms based on affective computing techniques, which are used to control audio-visual aspects of the game. The project is based on a fictitious Old Norse realm named ’Lundheim’, a place where emotions are woven into the fabric of reality. The game utilises Russell’s circumplex model of affect, providing four runes which correspond with different sections of the circumplex model. The player must activate each rune by entering the corresponding emotion state, which is captured using a consumer-grade Interaxon Muse electroencephalograph (EEG) headband. Activating each emotion triggers particle effects and corresponding sonic materials including interactive music, which are implemented with the Wwise video game audio middleware software. The project thereby demonstrates a novel implementation of affective technologies and sound in a video game, contributing towards discourses in this area of research.

Supplementary Material

MP4 File (Lundheim (Audio Mostly 2022).mp4)

Gameplay demonstration video

Download
167.46 MB

References

[1]

Ashwaq Alhargan, Neil Cooke, and Tareq Binjammaz. 2017. Affect recognition in an interactive gaming environment using eye tracking. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 285–291.

[2]

Rafael A Calvo and Sidney D’Mello. 2010. Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on affective computing 1, 1 (2010), 18–37.

Digital Library

[3]

Chuck-Jee Chau, Bin Wu, and Andrew Horner. 2015. The emotional characteristics and timbre of nonsustaining instrument sounds. Journal of the Audio Engineering Society 63, 4 (2015), 228–244.

[4]

Tuomas Eerola, Rafael Ferrer, and Vinoo Alluri. 2012. Timbre and affect dimensions: Evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds. Music Perception: An Interdisciplinary Journal 30, 1 (2012), 49–70.

[5]

Albert Ellis. 1957. Rational psychotherapy and individual psychology. Journal of individual psychology 13, 1 (1957), 38.

[6]

Tom Alexander Garner. 2013. Game Sound from Behind the Sofa: An Exploration into the Fear Potential of Sound and Psychophysiological Approaches to Audio-centric, Adaptive Gameplay. Ph. D. Dissertation.

[7]

Tom Alexander Garner. 2022. Biosensing technologies, biofeedback and Virtual Worlds. Special guest lecture at University of Greenwich, London, 18 March 2022.

[8]

Darryl Griffiths, Stuart Cunningham, Jonathan Weinel, and Richard Picking. 2021. A multi-genre model for music emotion recognition using linear regressors. Journal of New Music Research 50, 4 (2021), 355–372.

[9]

David Halbhuber, Jakob Fehle, Alexander Kalus, Konstantin Seitz, Martin Kocur, Thomas Schmidt, and Christian Wolff. 2019. The Mood Game - How to Use the Player’s Affective State in a Shoot’em up Avoiding Frustration and Boredom. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany) (MuC’19). Association for Computing Machinery, New York, NY, USA, 867–870. https://doi.org/10.1145/3340764.3345369

Digital Library

[10]

Christoffer Holmgård, Georgios N Yannakakis, Karen-Inge Karstoft, and Henrik Steen Andersen. 2013. Stress detection for ptsd via the startlemart game. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, 523–528.

Digital Library

[11]

Bojan Kerous, Filip Skola, and Fotis Liarokapis. 2018. EEG-based BCI and video games: a progress report. Virtual Reality 22, 2 (2018), 119–135.

Digital Library

[12]

Irene Kotsia, Stefanos Zafeiriou, and Spiros Fotopoulos. 2013. Affective gaming: A comprehensive survey. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 663–670.

Digital Library

[13]

Raúl Lara-Cabrera and David Camacho. 2019. A taxonomy and state of the art revision on affective games. Future Generation Computer Systems 92 (2019), 516–525.

Digital Library

[14]

John LaRocco, Minh Dong Le, and Dong-Guk Paeng. 2020. A systemic review of available low-cost EEG headsets used for drowsiness detection. Frontiers in neuroinformatics(2020), 42.

[15]

Tyler H McIntosh. 2021. Exploring the Relationship Between Music and Emotions with Machine Learning. Proceedings of EVA London 2021(2021), 279–280.

[16]

Faham Negini, Regan L Mandryk, and Kevin G Stanley. 2014. Using affective state to adapt characters, NPCs, and the environment in a first-person shooter game. In 2014 IEEE Games Media Entertainment. IEEE, 1–8.

[17]

Nintendo. 2017. The Legend of Zelda: Breath of the Wild. [Nintendo Switch].

[18]

Rosalind W Picard. 2000. Affective computing. MIT press.

[19]

Raquel Robinson, Katelyn Wiley, Amir Rezaeivahdati, Madison Klarkowski, and Regan L. Mandryk. 2020. ”Let’s Get Physiological, Physiological!”: A Systematic Review of Affective Gaming. Association for Computing Machinery, New York, NY, USA, 132–147. https://doi.org/10.1145/3410404.3414227

Digital Library

[20]

James A Russell. 1980. A circumplex model of affect.Journal of personality and social psychology 39, 6(1980), 1161.

[21]

Jonathan Sykes and Simon Brown. 2003. Affective Gaming: Measuring Emotion through the Gamepad. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 732–733. https://doi.org/10.1145/765891.765957

Digital Library

[22]

Jonathan Weinel. 2018. Inner sound: altered states of consciousness in electronic music and audio-visual media. Oxford University Press.

Cited By

Bhavsar PShah PSinha SKumar D(2024)Musical Neurofeedback Advancements, Feedback Modalities, and Applications: A Systematic ReviewApplied Psychophysiology and Biofeedback10.1007/s10484-024-09647-049:3(347-363)Online publication date: 5-Jun-2024
https://doi.org/10.1007/s10484-024-09647-0
Weinel J(2024)The Senses Beyond: New Directions in Game Engine ExperiencesThe Arts and Computational Culture: Real and Virtual Worlds10.1007/978-3-031-53865-0_21(499-520)Online publication date: 28-Jun-2024
https://doi.org/10.1007/978-3-031-53865-0_21
Mcintosh TWoscholski OBarthet M(2023)Affective Conditional Modifiers in Adaptive Video Game MusicProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616222(17-23)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3616195.3616222

Index Terms

Lundheim: Exploring Affective Audio Techniques in an Action-Adventure Video Game

Recommendations

Affective Conditional Modifiers in Adaptive Video Game Music
AM '23: Proceedings of the 18th International Audio Mostly Conference

This paper presents an application of affective conditional modifiers (ACMs) in adaptive video game music – a technique whereby the emotional intent of background music is adapted, based on biofeedback, to enforce a target emotion state in the player, ...
The influence of implicit and explicit biofeedback in first-person shooter games
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

To understand how implicit and explicit biofeedback work in games, we developed a first-person shooter (FPS) game to experiment with different biofeedback techniques. While this area has seen plenty of discussion, there is little rigorous ...
Affective Uplift During Video Game Play: A Naturalistic Case Study
Do video games affect players’ well-being? In this case study, we examined 162,325 intensive longitudinal in-game mood reports from 67,328 play sessions of 8,695 players of the popular game PowerWash Simulator. We compared players’ moods at the beginning ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AM '22: Proceedings of the 17th International Audio Mostly Conference

September 2022

245 pages

ISBN:9781450397018

DOI:10.1145/3561212

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

AM '22

AM '22: AudioMostly 2022

September 6 - 9, 2022

St. Pölten, Austria

Acceptance Rates

Overall Acceptance Rate 177 of 275 submissions, 64%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
144
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bhavsar PShah PSinha SKumar D(2024)Musical Neurofeedback Advancements, Feedback Modalities, and Applications: A Systematic ReviewApplied Psychophysiology and Biofeedback10.1007/s10484-024-09647-049:3(347-363)Online publication date: 5-Jun-2024
https://doi.org/10.1007/s10484-024-09647-0
Weinel J(2024)The Senses Beyond: New Directions in Game Engine ExperiencesThe Arts and Computational Culture: Real and Virtual Worlds10.1007/978-3-031-53865-0_21(499-520)Online publication date: 28-Jun-2024
https://doi.org/10.1007/978-3-031-53865-0_21
Mcintosh TWoscholski OBarthet M(2023)Affective Conditional Modifiers in Adaptive Video Game MusicProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616222(17-23)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3616195.3616222

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten