research-article

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances

Authors:

George Chernyshov,

Dingding Zheng,

Ragnar Thomsen,

Kouta Minamizawa,

Jamie A. WardAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 6, Issue 3

Article No.: 118, Pages 1 - 23

https://doi.org/10.1145/3550324

Published: 07 September 2022 Publication History

Abstract

Frisson is the feeling and experience of physical reactions such as shivers, tingling skin, and goosebumps. Using entrainment through facilitating interpersonal transmissions of embodied sensations, we present "Frisson Waves" with the aim to enhance live music performance experiences. "Frisson Waves" is an exploratory real-time system to detect, trigger and share frisson in a wave-like pattern over audience members during music performances. The system consists of a physiological sensing wristband for detecting frisson and a thermo-haptic neckband for inducing frisson. In a controlled environment, we evaluate detection (n=19) and triggering of frisson (n=15). Based on our findings, we conducted an in-the-wild music concert with 48 audience members using our system to share frisson. This paper summarizes a framework for accessing, triggering and sharing frisson. We report our research insights, lessons learned, and limitations of "Frisson Waves".

Supplementary Material

he (he.zip)

Supplemental movie, appendix, image and software files for, Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances

Download
269.16 MB

References

[1]

Josh Andres, MC Schraefel, Rakesh Patibanda, and Florian'Floyd' Mueller. 2020. Future InBodied: A Framework for Inbodied Interaction Design. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. 885--888.

Digital Library

[2]

Scott Bannister. 2019. Distinct varieties of aesthetic chills in response to multimedia. PloS one 14, 11 (2019), e0224974.

[3]

Scott Bannister. 2020. A vigilance explanation of musical chills? Effects of loudness and brightness manipulations. Music & Science 3 (2020), 2059204320915654.

[4]

Scott Bannister and Tuomas Eerola. 2018. Suppressing the chills: Effects of musical manipulation on the chills response. Frontiers in psychology 9 (2018), 2046.

[5]

Mathias Benedek and Christian Kaernbach. 2011. Physiological correlates and emotional specificity of human piloerection. Biological psychology 86, 3 (2011), 320--329.

[6]

Mathias Benedek, Barbara Wilfling, Reingard Lukas-Wolfbauer, Björn H Katzur, and Christian Kaernbach. 2010. Objective and continuous measurement of piloerection. Psychophysiology 47, 5 (2010), 989--993.

[7]

Giacomo Bignardi, Rebecca Chamberlain, Sofieke T Kevenaar, Zenab Tamimy, and Dorret I Boomsma. 2022. On the etiology of aesthetic chills: a behavioral genetic study. Scientific Reports 12, 1 (2022), 1--11.

[8]

Anne J Blood and Robert J Zatorre. 2001. Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. Proceedings of the national academy of sciences 98, 20 (2001), 11818--11823.

[9]

Roger Boldu, Mevan Wijewardena, Haimo Zhang, and Suranga Nanayakkara. 2020. MAGHair: A Wearable System to Create Unique Tactile Feedback by Stimulating Only the Body Hair. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. 1--10.

Digital Library

[10]

Wolfram Boucsein. 2012. Electrodermal activity. Springer Science & Business Media.

[11]

John T Cacioppo, Louis G Tassinary, and Gary Berntson. 2007. Handbook of psychophysiology. Cambridge university press.

[12]

Gheorghe Cernisov. 2020. An unobtrusive wearable sensing platform for psychological and social dynamics. Ph.D. Dissertation. Keio University.

[13]

Mitchell C Colver and Amani El-Alayli. 2016. Getting aesthetic chills from music: The connection between openness to experience and frisson. Psychology of Music 44, 3 (2016), 413--427.

[14]

Ella A Cooper, John Garlick, Eric Featherstone, Valerie Voon, Tania Singer, Hugo D Critchley, and Neil A Harrison. 2014. You turn me cold: evidence for temperature contagion. PloS one 9, 12 (2014), e116126.

[15]

Michael E Dawson, Anne M Schell, and Diane L Filion. 2017. The electrodermal system. (2017).

[16]

Rémi de Fleurian and Marcus Pearce. 2020. Chills in music: An integrative review. PsyArXiv 1 (2020), 1--50.

[17]

Shogo Fukushima and Hiroyuki Kajimoto. 2012. Facilitating a surprised feeling by artificial control of piloerection on the forearm. In Proceedings of the 3rd Augmented Human International Conference. 1--4.

Digital Library

[18]

Shkurta Gashi, Elena Di Lascio, and Silvia Santini. 2019. Using unobtrusive wearable sensors to measure the physiological synchrony between presenters and audience members. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (2019), 1--19.

Digital Library

[19]

Oliver Grewe, Björn Katzur, Reinhard Kopiez, and Eckart Altenmüller. 2011. Chills in different sensory domains: Frisson elicited by acoustical, visual, tactile and gustatory stimuli. Psychology of Music 39, 2 (2011), 220--239.

[20]

Oliver Grewe, Reinhard Kopiez, and Eckart Altenmüüller. 2009. The chill parameter: Goose bumps and shivers as promising measures in emotion research. Music Perception 27, 1 (2009), 61--74.

[21]

Yan He, George Chernyshov, Dingding Zheng, Jiawen Han, Ragnar Thomsen, Danny Hynds, Yuehui Yang, Yun Suen Pai, Kai Kunze, and Kouta Minamizawa. 2021. Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances. In SIGGRAPH Asia 2021 Emerging Technologies. 1--2.

[22]

Y He, D Zheng, G Chernyshov, R Thomsen, J Han, D Hynds, Y S Pai, K Kunze, and Kouta Minamizawa. 2021. Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances. In 2021 IEEE World Haptics Conference (WHC). IEEE, 591--591.

[23]

Yuri Ishikawa, Anzu Kawazoe, George Chernyshov, Shinya Fujii, and Masashi Nakatani. 2018. The Thermal Feedback Influencer: Wearable Thermal Display for Enhancing the Experience of Music Listening. In International AsiaHaptics conference. Springer, 162--168.

[24]

Abhinandan Jain, Adam Haar Horowitz, Felix Schoeller, Sang-won Leigh, Pattie Maes, and Misha Sra. 2020. Designing Interactions Beyond Conscious Control: A New Model for Wearable Interfaces. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--23.

Digital Library

[25]

Peter E Keller, Giacomo Novembre, and Michael J Hove. 2014. Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philosophical Transactions of the Royal Society B: Biological Sciences 369, 1658 (2014), 20130394.

[26]

Jaemin Kim, Dae Geon Seo, and Young-Ho Cho. 2015. Capacitive Skin Piloerection Sensors for Human Emotional State Cognition. Transactions of the Korean Society of Mechanical Engineers B 39, 2 (2015), 147--152.

[27]

Josephine Klefeker, Libi Striegl, and Laura Devendorf. 2020. What HCI Can Learn from ASMR: Becoming Enchanted with the Mundane. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[28]

Stefan Koelsch. 2014. Brain correlates of music-evoked emotions. Nature Reviews Neuroscience 15, 3 (2014), 170--180.

[29]

Dominique Makowski, Tam Pham, Zen J. Lau, Jan C. Brammer, François Lespinasse, Hung Pham, Christopher Schölzel, and S. H. Annabel Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior Research Methods (02 Feb 2021). https://doi.org/10.3758/s13428-020-01516-y

[30]

Jonathon McPhetres. 2021. The physiological study of emotional piloerection: A review and guide for future research. (2021).

[31]

Kazuma Mori and Makoto Iwanaga. 2017. Two types of peak emotional responses to music: The psychophysiology of chills and tears. Scientific reports 7, 1 (2017), 1--13.

[32]

Kristin Neidlinger, Lianne Toussaint, Edwin Dertien, Khiet P Truong, Hermie Hermens, and Vanessa Evers. 2019. Emotional prosthesis for animating awe through performative biofeedback. In Proceedings of the 23rd International Symposium on Wearable Computers. 312--317.

Digital Library

[33]

Kristin Neidlinger, Khiet P Truong, Caty Telfair, Loe Feijs, Edwin Dertien, and Vanessa Evers. 2017. AWElectric: that gave me goosebumps, did you feel it too?. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 315--324.

Digital Library

[34]

Shuo Niu, Hugh S Manon, Ava Bartolome, Nguyen Binh Ha, and Keegan Veazey. 2022. Close-up and Whispering: An Understanding of Multimodal and Parasocial Interactions in YouTube ASMR videos. In CHI Conference on Human Factors in Computing Systems. 1--18.

Digital Library

[35]

Emily C Nusbaum and Paul J Silvia. 2011. Shivers and timbres: Personality and the experience of chills from music. Social Psychological and Personality Science 2, 2 (2011), 199--204.

[36]

Kelsey E Onderdijk, Dana Swarbrick, Bavo Van Kerrebroeck, Maximillian Mantei, Jonna K Vuoskoski, Pieter-Jan Maes, and Marc Leman. 2021. Livestream experiments: the role of COVID-19, agency, presence, and social context in facilitating social connectedness. Frontiers in psychology 12 (2021), 1741.

[37]

Richard V Palumbo, Marisa E Marraccini, Lisa L Weyandt, Oliver Wilder-Smith, Heather A McGee, Siwei Liu, and Matthew S Goodwin. 2017. Interpersonal autonomic physiology: A systematic review of the literature. Personality and Social Psychology Review 21, 2 (2017), 99--141.

[38]

Jaak Panksepp. 1995. The emotional sources of" chills" induced by music. Music perception 13, 2 (1995), 171--207.

[39]

Matthew Pelowski, Helmut Leder, Vanessa Mitschke, Eva Specker, Gernot Gerger, Pablo PL Tinio, Elena Vaporova, Till Bieg, and Agnes Husslein-Arco. 2018. Capturing aesthetic experiences with installation art: An empirical assessment of emotion, evaluations, and mobile eye tracking in Olafur Eliasson's "Baroque, Baroque!". Frontiers in Psychology 9 (2018), 1255.

[40]

Leonid Perlovsky. 2014. Aesthetic emotions, what are their cognitive functions? Frontiers in Psychology 5 (2014), 98.

[41]

Ming-Zher Poh. 2011. Continuous assessment of epileptic seizures with wrist-worn biosensors. Ph.D. Dissertation. Massachusetts Institute of Technology.

[42]

Matthew E Sachs, Robert J Ellis, Gottfried Schlaug, and Psyche Loui. 2016. Brain connectivity reflects human aesthetic responses to music. Social cognitive and affective neuroscience 11, 6 (2016), 884--891.

[43]

Alfonsina Scarinzi. 2015. Aesthetics and the embodied mind: beyond art theory and the cartesian mind-body dichotomy. Springer.

[44]

Klaus R Scherer and Eduardo Coutinho. 2013. How music creates emotion: a multifactorial process approach. The emotional power of music (2013), 121--145.

[45]

Ines Schindler, Georg Hosoya, Winfried Menninghaus, Ursula Beermann, Valentin Wagner, Michael Eid, and Klaus R Scherer. 2017. Measuring aesthetic emotions: A review of the literature and a new assessment tool. PloS one 12, 6 (2017), e0178899.

[46]

Félix Schoeller, AJH Haar, Abhinandan Jain, and Pattie Maes. 2019. Enhancing human emotions with interoceptive technologies. Physics of life reviews 31 (2019), 310--319.

[47]

Felix Schoeller and Leonid Perlovsky. 2016. Aesthetic chills: Knowledge-acquisition, meaning-making, and aesthetic emotions. Frontiers in psychology 7 (2016), 1093.

[48]

Fred Shaffer and Jay P Ginsberg. 2017. An overview of heart rate variability metrics and norms. Frontiers in public health (2017), 258.

[49]

Lin Shu, Jinyan Xie, Mingyue Yang, Ziyi Li, Zhenqi Li, Dan Liao, Xiangmin Xu, and Xinyi Yang. 2018. A review of emotion recognition using physiological signals. Sensors 18, 7 (2018), 2074.

[50]

Paul J Silvia and Emily C Nusbaum. 2011. On personality and piloerection: Individual differences in aesthetic chills and other unusual aesthetic experiences. Psychology of Aesthetics, Creativity, and the Arts 5, 3 (2011), 208.

[51]

John A Sloboda. 1991. Music structure and emotional response: Some empirical findings. Psychology of music 19, 2 (1991), 110--120.

[52]

Robert Stecker. 2006. Aesthetic experience and aesthetic value. Philosophy Compass 1, 1 (2006), 1--10.

[53]

Dana Swarbrick, Dan Bosnyak, Steven R Livingstone, Jotthi Bansal, Susan Marsh-Rollo, Matthew H Woolhouse, and Laurel J Trainor. 2019. How live music moves us: head movement differences in audiences to live versus recorded music. Frontiers in psychology 9 (2019), 2682.

[54]

Wolfgang Tschacher, Steven Greenwood, Hauke Egermann, Melanie Wald-Fuhrmann, Anna Czepiel, Martin Tröndle, and Deborah Meier. 2021. Physiological synchrony in audiences of live concerts. Psychology of aesthetics, creativity, and the arts (2021).

[55]

Daniel T Van Bel, Karin CHJ Smolders, Wijnand A IJsselsteijn, and YAW De Kort. 2009. Social connectedness: concept and measurement. In Intelligent Environments 2009. IOS Press, 67--74.

[56]

Robert J Zatorre and Valorie N Salimpoor. 2013. From perception to pleasure: music and its neural substrates. Proceedings of the National Academy of Sciences 110, Supplement 2 (2013), 10430--10437.

[57]

Kejun Zhang, Hui Zhang, Simeng Li, Changyuan Yang, and Lingyun Sun. 2018. The pmemo dataset for music emotion recognition. In Proceedings of the 2018 acm on international conference on multimedia retrieval. 135--142.

Digital Library

Cited By

Feng FBennett DMekler E(2024)Smiles Summon the Warmth of Spring: A Design Framework for Thermal-Affective Interaction based in Chinese Cí PoetryProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661620(2802-2819)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661620
He KLi CDuan YFeng JZhou J(2024)TrackPoseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314597:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631459
Huang ZCao XZhang YMa X(2024)Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic ChillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642818(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642818
Show More Cited By

Index Terms

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances
1. Human-centered computing

Recommendations

Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances
SA '21: SIGGRAPH Asia 2021 Emerging Technologies

Frisson is a feeling and a mental experience of body reactions such as shivers, tingling skin, and goosebumps. However, this sensation is not shareable naturally with others and is rarely used in live performances. We propose Frisson Waves, a real-time ...
REVIVE: An Audio-visual Performance with Musical and Visual AI Agents
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

REVIVE explores the affordances of live interaction between the artificial musical agent MASOM, human electronic musicians, and visual generation agents. The Musical Agent based on Self-Organizing Maps (MASOM) has memorized sound objects and learned how ...
Video conducting the olympic games 2008: the iTV field trial of the EU-IST project LIVE
DIMEA '08: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts

The EU IP project LIVE investigates novel iTV formats, production methods and tools for staging of live media events in a professional broadcast environment. The vision of LIVE is to change the role of the traditional live TV Director to become a future ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 6, Issue 3

September 2022

1612 pages

EISSN:2474-9567

DOI:10.1145/3563014

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2022

Published in IMWUT Volume 6, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
581
Total Downloads

Downloads (Last 12 months)213
Downloads (Last 6 weeks)32

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Feng FBennett DMekler E(2024)Smiles Summon the Warmth of Spring: A Design Framework for Thermal-Affective Interaction based in Chinese Cí PoetryProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661620(2802-2819)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661620
He KLi CDuan YFeng JZhou J(2024)TrackPoseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314597:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631459
Huang ZCao XZhang YMa X(2024)Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic ChillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642818(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642818
Duan YYu JFeng JHe KLu JZhou J(2023)3D Finger Rotation Estimation from Fingerprint ImagesProceedings of the ACM on Human-Computer Interaction10.1145/36264677:ISS(114-134)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626467
Torun MMostofi Y(2023)Wi-FlexProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109307:3(1-27)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610930
Meng XHe YKunze KWard JMcGill MMarky K(2023)Towards Enhancing a Recorded Concert Experience in Virtual Reality by Visualizing the Physiological Data of the AudienceProceedings of the Augmented Humans International Conference 202310.1145/3582700.3583709(330-333)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3583709
Fernandez AKim CGunasekaran TPai YMinamizawa K(2023)Virtual Journalist: Measuring and Inducing Cultural Empathy by Visualizing Empathic Perspectives in VR2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00143(667-672)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00143

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents