research-article

Enhancing Audience Engagement in Performing Arts Through an Adaptive Virtual Environment with a Brain-Computer Interface

Authors:

Tianyu HuangAuthors Info & Claims

IUI '16: Proceedings of the 21st International Conference on Intelligent User Interfaces

Pages 306 - 316

https://doi.org/10.1145/2856767.2856768

Published: 07 March 2016 Publication History

Abstract

Audience engagement is an important indicator of the quality of the performing arts but hard to measure. Psychophysiological measurements are promising research methods for perceiving and understanding audience's responses in real-time. Currently, such research are conducted by collecting biometric data from audience when they are watching a performance. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from quality of performing arts to develop a system that monitor audience engagement in real time using electroencephalography (EEG) measurement and seek to improve it by triggering the adaptive performing cues when the engagement level decreased. We simulated the immersive theatre performances to provide audience a high-fidelity visual-audio experience. An experimental evaluation is conducted with 48 participants during two different performance studies. The results showed that our system could successfully detect the decreases in audience engagement and the performing cues had positive effects on regain audience engagement. Our research offers the guidelines for designing theatre performances from the audience's perception.

Supplementary Material

suppl.mov (iuifp0110-file3.mp4)

Supplemental video

Download
71.99 MB

References

[1]

Andujar, M., and Gilbert, J. E. Let's learn! Enhancing User's Engagement Levels Through Passive BrainComputer Interfaces. Ext. Abstracts CHI 2013, ACM Press (2013), 703--708.

Digital Library

[2]

Ayaz, H., Shewokis, P., Bunce, S., Schultheis, M., and Onaral, B. Assessment of cognitive neural correlates for a functional near infrared-based brain computer interface system. In Augmented Cognition, HCII 2009, vol. 5638 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg, 2009, 699--708.

Digital Library

[3]

Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T, Olmstead, R. E., Tremoulet, P. L. EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation, Space, and Environment Medicine 78, Supplement 1(5), B231--B244.

[4]

Blankertz, B., Dornhege, G., Krauledat, M., Muller, K. and Curio, G. The non-invasive Berlin Brain-Computer Interface: Fast acquisition of effective performance in untrained subjects. NeuroImage. 37, 2:539--550.

[5]

Bouscein, W. 1992. Electrodermal Activity. Plenum Press, New York, USA.

[6]

Brefczynski, J. A., and DeYoe, E. A. A physiological correlate of the 'spotlight' of visual attention. Nature Neuroscience, Volume 2, 1999, 370--374.

[7]

Calvo-Merino, B., Jola, C., Glaser, D.E., Haggard, P. 2008. Towards a sensorimotor aesthetics of performing art. Consciousness and Cognition. 17, 3: 911--922.

[8]

Cruse, W. 2014. Stagecraft theatre. ENCYCLOPAEDIA BRITANNICA. http://global.britannica.com/art/stagecraft#toc40210

[9]

Cutrell, E., Tan, D. BCI for passive input in HCI. In Proc CHI'07, 2007.

[10]

Dixon, S. 2015. Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation. The MIT Press, 2015.

[11]

Ferrez, P. W., and Millan. You are wrong! Automatic detection of interaction errors from brain waves. In Proc IJCAI'05 (2005).

Digital Library

[12]

George, L., and Lecuyer, A. An overview of research on "passive" brain-computer interfaces for implicit human-computer interaction. International Conference on Applied Bionics and Biomechanics ICABB 2010, workshop "Brain-Computer Interfacing and Virtual Reality", Oct 2010.

[13]

Girouard, A., Solovey, E. T., and Jacob, R. J. K. Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy. International Journal of Autonomous and Adaptive Communications Systems (2010).

Digital Library

[14]

Grant, C. 2013. Many Worlds: The movie that watches its audience. BBC News http://www.bbc.co.uk/news/technology-21429437

[15]

Huang, J., Yu, C., Wang, Y. T., Zhao, Y. H., Liu, SQ., Mo, C., Liu, J., Zhang, L., and Shi, Y. C. 2014. FOCUS: enhancing children's engagement in reading by using contextual BCI training sessions. In Proc CHI'14, 1905--1908.

Digital Library

[16]

Jasper, H. H. 1958. The ten-twenty electrode system of the International Federation. Electroencephalography and Clinical Neurophysiology, 371--375.

[17]

Jatupaiboon, N., Pan-ngum, S., and Israsena, P. 2013. Real-Time EEG-Based Happiness Detection System. The Scientific World Journal, Volume 2013, 618649.

[18]

Latulipe, C., Carroll, Erin A., and Lottridge, D. 2011. Love, hate, arousal and engagement: exploring audience responses to performing arts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 1845--1854.

Digital Library

[19]

Lang, P. 1995. The emotion probe: Studies of motivation and attention. American Psychologist. 50(5): 372--385.

[20]

Lee, J. C., and Tan, D. S. 2006. Using a low-cost electroencephalograph for task classification in HCI research. In Proc UIST'06, 81--90.

Digital Library

[21]

Molina, G., Tsoneva, T., and Nijholt, A. Emotional brain-computer interfaces. In Proc ACII'09, IEEE (2009), 1--9.

[22]

Napoli, R., Gloman, C. 2006. Scenic Design and Lighting Techniques: A Basic Guide for Theatre. Focal Press, 2006.

[23]

Niedermeyer, E., and F. H. Lopes Da Silva. 1993. Electroencephalography: Basic principles, clinical applications and related fields, 3rd edition, Lippincott, Williams & Wilkins, Philadelphia.

[24]

Nijholt, A., Bos, D. P.-O., Reuderink, B. Turning shortcomings into challenges: Brain-computer interfaces for games. Entertainment Computing 1, 2 (2009), 85--94.

[25]

Pfurtscheller, G., Neuper, C., Guger, C., Harkam, W., Ramoser, H., Schlogl, A., Obermaier, B., and Pregenzer, M. Current trends in Graz Brain-Computer (BCI) research. IEEE Transactions on Rehabilitation Engineering 8, 2 (2000), 216--219.

[26]

Pope, A.T., Bogart, E.H., Bartolome, D.S. 1995. Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology. 40(1995), 187--195.

[27]

Radbourne, J., Johanson, K., Glow, H., and White, T. 2009. The audience experience: measuring quality in the performing arts. International journal of Arts Management 11, 3:16--29.

[28]

Radbourne, J., Glow, H. and Johanson, K. 2010. Hidden stories: listening to the audience at the live performance. Double dialogues, no.13, summer, 1--14.

[29]

Santoro, C., and G. Troilo. 2007. The Drivers of Hedonic Consumption Experience: A semiotic analysis of rock concerts. In Consuming Experience. Abingdon, UK, and New York: Routledge.

[30]

Szafir, D. and Mutlu, B. 2012. Pay attention! Designing adaptive agents that monitor and improve user engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 11--20.

Digital Library

[31]

Tan, D., Nijholt, A. 2010. Brain-computer interface: Applying our minds to human-computer interaction. Springer, 2010.

Digital Library

[32]

Tan, D. Brain-computer interfaces: applying our minds to human-computer interaction. In Proc CHI Workshop: "What is the Next Generation of HumanComputer Interaction?", 2006.

[33]

Teplan, M. Fundamentals of EEG Measurement. Measurement Science Review, vol 2, section 2, 2002, 1--11.

[34]

Wang, C., Geelhoed, Erik N., Stenton, Phil, P., and Cesar, P. 2014. Sensing a live audience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1909--1912.

Digital Library

[35]

Zander, T., and Kothe, C. Towards passive braincomputer interfaces: applying brain-computer interface technology to human-machine systems in general. Journal of Neural Engineer 8, 2(2011), 025005.

Cited By

Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309
Vanutelli MSalvadore MLucchiari C(2023)BCI Applications to Creativity: Review and Future Directions, from little-c to C2Brain Sciences10.3390/brainsci1304066513:4(665)Online publication date: 15-Apr-2023
https://doi.org/10.3390/brainsci13040665
Han JChernyshov GSugawa MZheng DHynds DFurukawa TPadovani Macieira MMarky KMinamizawa KWard JKunze K(2023)Linking Audience Physiology to ChoreographyACM Transactions on Computer-Human Interaction10.1145/355788730:1(1-32)Online publication date: 7-Mar-2023
https://dl.acm.org/doi/10.1145/3557887
Show More Cited By

Index Terms

Enhancing Audience Engagement in Performing Arts Through an Adaptive Virtual Environment with a Brain-Computer Interface
1. Applied computing
  1. Arts and humanities
    1. Performing arts
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Exploring Audience Response in Performing Arts with a Brain-Adaptive Digital Performance System
Special Issue on IUI 2016 Highlights

Audience response is an important indicator of the quality of performing arts. Psychophysiological measurements enable researchers to perceive and understand audience response by collecting their bio-signals during a live performance. However, how the ...
Towards ambulatory brain-computer interfaces: a pilot study with P300 signals
ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment Technology

Brain-Computer Interfaces (BCI) are communication systems that enable users to interact with computers using only brain activity. This activity is generally measured by ElectroEncephaloGraphy (EEG). A major limitation of BCI is the electrical ...
Enhancing Audience Engagement Through Immersive 360-Degree Videos: An Experimental Study
Augmented Cognition: Intelligent Technologies
Abstract
The use of 360-degree videos to engage audiences in diverse contexts is increasing. While 360-degree videos have the potential to create new value in enhancing audiences’ viewing experiences, they often decrease audience engagement by causing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '16: Proceedings of the 21st International Conference on Intelligent User Interfaces

March 2016

446 pages

ISBN:9781450341370

DOI:10.1145/2856767

General Chairs:
Jeffrey Nichols
Google Inc, USA
,
Jalal Mahmud
IBM Research, USA
,
John O'Donovan
UC Santa Barbara, USA
,
Program Chairs:
Cristina Conati
University of British Columbia, Canada
,
Massimo Zancanaro
Bruno Kessler Foundation, Italy

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

IUI'16

Sponsor:

IUI'16: 21st International Conference on Intelligent User Interfaces

March 7 - 10, 2016

California, Sonoma, USA

Acceptance Rates

IUI '16 Paper Acceptance Rate 49 of 194 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
816
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)5

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309
Vanutelli MSalvadore MLucchiari C(2023)BCI Applications to Creativity: Review and Future Directions, from little-c to C2Brain Sciences10.3390/brainsci1304066513:4(665)Online publication date: 15-Apr-2023
https://doi.org/10.3390/brainsci13040665
Han JChernyshov GSugawa MZheng DHynds DFurukawa TPadovani Macieira MMarky KMinamizawa KWard JKunze K(2023)Linking Audience Physiology to ChoreographyACM Transactions on Computer-Human Interaction10.1145/355788730:1(1-32)Online publication date: 7-Mar-2023
https://dl.acm.org/doi/10.1145/3557887
Putze FPutze SSagehorn MMicek CSolovey E(2022)Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and ReuseACM Transactions on Computer-Human Interaction10.1145/349055429:4(1-43)Online publication date: 31-Mar-2022
https://dl.acm.org/doi/10.1145/3490554
Chen XNiu LVeeraraghavan ASabharwal A(2022)FaceEngage: Robust Estimation of Gameplay Engagement from User-Contributed (YouTube) VideosIEEE Transactions on Affective Computing10.1109/TAFFC.2019.294501413:2(651-665)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TAFFC.2019.2945014
Ge SLuo SYan SShen X(2022)Effects of Augmenting Real-Time Biofeedback in An Immersive VR Performance2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00159(751-756)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00159
Elagroudy PWang XStemasov EHirzle TShishkovets SMehrotra SSchmidt AZhou BCheng JStrohmeier PIshimaru SKosch T(2020)SpotlessMindProceedings of the Augmented Humans International Conference10.1145/3384657.3384800(1-8)Online publication date: 16-Mar-2020
https://dl.acm.org/doi/10.1145/3384657.3384800
Damiano RLombardo VMonticone GPizzo A(2020)Studying and designing emotions in live interactions with the audienceMultimedia Tools and Applications10.1007/s11042-020-10007-3Online publication date: 20-Oct-2020
https://doi.org/10.1007/s11042-020-10007-3
Karatas NSekino HTanaka T(2020)Utilization of Human-Robot Interaction for the Enhancement of Performer and Audience Engagement in Performing ArtHCI International 2020 – Late Breaking Papers: Interaction, Knowledge and Social Media10.1007/978-3-030-60152-2_26(348-358)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-60152-2_26
Kosmyna NMaes P(2019)AttentivU: An EEG-Based Closed-Loop Biofeedback System for Real-Time Monitoring and Improvement of Engagement for Personalized LearningSensors10.3390/s1923520019:23(5200)Online publication date: 27-Nov-2019
https://doi.org/10.3390/s19235200
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten