Impact of the Nonverbal Behavior of Virtual Audience on Users' Perception of Social Attitudes
Authors: 
Marion Ristorcelli, Alexandre D'Ambra, Jean-Marie Pergandi, Rémy Casanova, Magalie OchsAuthors Info & Claims
Marion Ristorcelli, Alexandre D'Ambra, Jean-Marie Pergandi, Rémy Casanova, Magalie OchsAuthors Info & ClaimsArticle No.: 16, Pages 1 - 9
Abstract
In a virtual reality public speaking training system, it is essential to control the audience’s nonverbal behavior in order to simulate different attitudes. The virtual audience’s social attitude is generally represented by a two-dimensional valence-arousal model describing the opinion and engagement of virtual characters. In this article, we argue that the valence-arousal representation is not sufficient to describe the user’s perception of a virtual character’s social attitude. We propose a three-dimensional model by dividing the valence axis into two dimensions representing the epistemic and affective stance of the virtual character, reflecting the character’s agreement and emotional reaction. To assess the perception of the virtual characters’ nonverbal behavior on these two new dimensions, we conducted a perceptive study in virtual reality with 44 participants who evaluated 50 animations combining multimodal nonverbal behavioral signals such as head movements, facial expressions, gaze direction and body posture. The results of our experiment show that, in fact, the valence axis should be divided into two axes to take into account the perception of the virtual character’s epistemic and affective stance. Furthermore, the results show that one behavioral signal is predominant for the evaluation of each dimension: head movements for the epistemic dimension and facial expressions for the affective dimension. These results provide useful guidelines for designing the nonverbal behavior of a virtual audience for social attitudes’ simulation.
References
[1]
Ligia Batrinca, Giota Stratou, Ari Shapiro, Louis-Philippe Morency, and Stefan Scherer. 2013. Cicero-towards a multimodal virtual audience platform for public speaking training. In Intelligent Virtual Agents: 13th International Conference, IVA 2013, Edinburgh, UK, August 29-31, 2013. Proceedings 13. Springer, 116–128.
[2]
Daniel J Benjamin and James O Berger. 2019. Three recommendations for improving the use of p-values. The American Statistician 73, sup1 (2019), 186–191.
[3]
Pio E Ricci Bitti, Luisa Bonfiglioli, Paolo Melani, Roberto Caterina, and Pierluigi Garotti. 2014. Expression and communication of doubt/uncertainty through facial expression. Ricerche di Pedagogia e Didattica. Journal of Theories and Research in Education 9, 1 (2014), 159–177.
[4]
Massimo Chindamo, Jens Allwood, and Elisabeth Ahlsén. 2012. Some suggestions for the study of stance in communication. In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing. IEEE, 617–622.
[5]
Mathieu Chollet and Stefan Scherer. 2017. Perception of virtual audiences. IEEE computer graphics and applications 37, 4 (2017), 50–59.
[6]
Mathieu Chollet, Giota Sratou, Ari Shapiro, Louis-Philippe Morency, and Stefan Scherer. 2014. An interactive virtual audience platform for public speaking training. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. 1657–1658.
[7]
Bianca Malaika Ciuffani. 2017. Non-verbal Communication and Leadership: the impact of hand gestures used by leaders on follower job satisfaction. B.S. thesis. University of Twente.
[8]
Fiona Dermody and Alistair Sutherland. 2018. Multimodal systems for public speaking: a case in support of a positive computing approach. In Proceedings of the 2nd International Conference on Computer-Human Interaction Research and Applications-Volume 1: CHIRA, Vol. 1. Scitepress, 170–175.
[9]
John P Doody and Peter Bull. 2011. Asperger’s syndrome and the decoding of boredom, interest, and disagreement from body posture. Journal of nonverbal behavior 35 (2011), 87–100.
[10]
Mariia Dubiago, Sandra Poeschl, and Nicola Doering. 2017. Influence of Simulation Fidelity on Perceived Simulation Realism–An Exploratory Study on a Virtual Public Speaking Training Application. Annu. Rev. CyberTher. Telemed 15 (2017), 154–159.
[11]
Meriem El-Yamri, Alejandro Romero-Hernandez, Manuel Gonzalez-Riojo, and Borja Manero. 2019. Designing a VR game for public speaking based on speakers features: a case study. Smart Learning Environments 6, 1 (2019), 12.
[12]
Paul MG Emmelkamp, Katharina Meyerbröker, and Nexhmedin Morina. 2020. Virtual reality therapy in social anxiety disorder. Current psychiatry reports 22 (2020), 1–9.
[13]
Elodie Etienne, Anne-Lise Leclercq, Angélique Remacle, Laurence Dessart, and Michaël Schyns. 2023. Perception of avatars nonverbal behaviors in virtual reality. Psychology & Marketing 40, 11 (2023), 2464–2481.
[14]
Farina Freigang, Sören Klett, and Stefan Kopp. 2017. Pragmatic multimodality: Effects of nonverbal cues of focus and certainty in a virtual human. In Intelligent Virtual Agents: 17th International Conference, IVA 2017, Stockholm, Sweden, August 27-30, 2017, Proceedings 17. Springer, 142–155.
[15]
Yann Glémarec, Jean-Luc Lugrin, Anne-Gwenn Bosser, Cédric Buche, and Marc Erich Latoschik. 2022. Controlling the stage: a high-level control system for virtual audiences in Virtual Reality. Frontiers in Virtual Reality 3 (2022), 876433.
[16]
Yann Glémarec, Jean-Luc Lugrin, Anne-Gwenn Bosser, Aryana Collins Jackson, Cédric Buche, and Marc Erich Latoschik. 2021. Indifferent or enthusiastic? virtual audiences animation and perception in virtual reality. Frontiers in Virtual Reality 2 (2021), 666232.
[17]
Bin Han and Gerard Jounghyun Kim. 2021. Audiencemr: Extending the local space for large-scale audience with mixed reality for enhanced remote lecturer experience. Applied Sciences 11, 19 (2021), 9022.
[18]
Michał Jakubowski, Marcin Wardaszko, Anna Winniczuk, Błażej Podgórski, and Małgorzata Ćwil. 2019. Cicero VR-Public Speaking Training Tool and an Attempt to Create Positive Social VR Experience. In Virtual, Augmented and Mixed Reality. Applications and Case Studies: 11th International Conference, VAMR 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Part II 21. Springer, 297–311.
[19]
Andrew F Jarosz and Jennifer Wiley. 2014. What are the odds? A practical guide to computing and reporting Bayes factors. The Journal of Problem Solving 7, 1 (2014), 2.
[20]
Harold Jeffreys. 1998. The theory of probability. OuP Oxford.
[21]
Ni Kang, Willem-Paul Brinkman, M Birna Van Riemsdijk, and Mark Neerincx. 2016. The design of virtual audiences: noticeable and recognizable behavioral styles. Computers in Human Behavior 55 (2016), 680–694.
[22]
Robert E Kass and Adrian E Raftery. 1995. Bayes factors. Journal of the american statistical association 90, 430 (1995), 773–795.
[23]
Emmanuelle P Kleinlogel, Marion Curdy, João Rodrigues, Carmen Sandi, and Marianne Schmid Mast. 2021. Doppelganger-based training: Imitating our virtual self to accelerate interpersonal skills learning. PloS one 16, 2 (2021), e0245960.
[24]
Nicolas Mascret, Olivier Vors, Tanguy Marqueste, Rémy Casanova, and François Cury. 2019. Social support from evaluative familiar persons–a buffer against stress? Preliminary evidence of neuroendocrine and psychological responses. Anxiety, Stress, & Coping 32, 5 (2019), 534–544.
[25]
Emma Otta, Beatriz Barcellos Pereira Lira, Nadia Maria Delevati, Otávio Pimentel Cesar, and Carla Salati Guirello Pires. 1994. The effect of smiling and of head tilting on person perception. The Journal of psychology 128, 3 (1994), 323–331.
[26]
Fabrizio Palmas, Jakub Cichor, David A Plecher, and Gudrun Klinker. 2019. Acceptance and effectiveness of a virtual reality public speaking training. In 2019 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 363–371.
[27]
Fabrizio Palmas, Ramona Reinelt, Jakub E Cichor, David A Plecher, and Gudrun Klinker. 2021. Virtual reality public speaking training: Experimental evaluation of direct feedback technology acceptance. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 463–472.
[28]
David-Paul Pertaub, Mel Slater, and Chris Barker. 2002. An experiment on public speaking anxiety in response to three different types of virtual audience. Presence 11, 1 (2002), 68–78.
[29]
Jakub Procházka, Daniel Kerekeš, 2017. Influence of autonomy and task significance on engagement: Experiment. In Trends and Issues in Interdisciplinary Behavior and Social Science: Proceedings of the 5th International Congress on Interdisciplinary Behavior and Social Science (ICIBSoS 2016), 5-6 November 2016, Jogjakarta, Indonesia. CRC Press, 21–24.
[30]
Marion Ristorcelli, Emma Gallego, Kévin Nguy, Jean-Marie Pergandi, Rémy Casanova, and Magalie Ochs. 2023. Investigating the impact of a virtual audience’s gender and attitudes on a human speaker. In Companion Publication of the 25th International Conference on Multimodal Interaction. 363–367.
[31]
Christian Seufert, Sebastian Oberdörfer, Alice Roth, Silke Grafe, Jean-Luc Lugrin, and Marc Erich Latoschik. 2022. Classroom management competency enhancement for student teachers using a fully immersive virtual classroom. Computers & Education 179 (2022), 104410.
[32]
Carolin Straßmann, Astrid Rosenthal von der Pütten, Ramin Yaghoubzadeh, Raffael Kaminski, and Nicole Krämer. 2016. The effect of an intelligent virtual agent’s nonverbal behavior with regard to dominance and cooperativity. In Intelligent Virtual Agents: 16th International Conference, IVA 2016, Los Angeles, CA, USA, September 20–23, 2016, Proceedings 16. Springer, 15–28.
[33]
Robin E Sülter, Paul E Ketelaar, and Wolf-Gero Lange. 2022. SpeakApp-Kids! Virtual reality training to reduce fear of public speaking in children–A proof of concept. Computers & Education 178 (2022), 104384.
[34]
BK Wiederhold and G Riva. 2012. Designing virtual audiences for fear of public speaking training–an observation study on realistic nonverbal behavior. Annu. Rev. Cybertherapy Telemed. Adv. Technol. Behav. Soc. Neurosci 181 (2012), 218.
[35]
Haipeng Zeng, Xingbo Wang, Yong Wang, Aoyu Wu, Ting-Chuen Pong, and Huamin Qu. 2022. Gesturelens: Visual analysis of gestures in presentation videos. IEEE Transactions on Visualization and Computer Graphics (2022).
Index Terms
- Impact of the Nonverbal Behavior of Virtual Audience on Users' Perception of Social Attitudes
Recommendations
Investigating the Impact of a Virtual Audience's Gender and Attitudes on a Human Speaker
ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal InteractionPublic speaking anxiety is a common social phobia in the general population, particularly among young adults. This social skill is essential for professional integration. However, few solutions have been developed to train people in public speaking. The ...
Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video
AbstractThis research analyzes the psychological reactions of participants to the neutral, positive, or negative attitudes of a virtual audience recorded in 360°-video. Participants were asked to deliver three speeches, each accompanied by a different ...
Model of the perception of smiling virtual character
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1A smile may convey different communicative intentions depending on subtle characteristics of the facial expression. Moreover, during an interaction, the expression of smile impacts on the observer's perception of both the social stance of the speaker ...
Comments
Information & Contributors
Information
Published In
June 2024
578 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AVI 2024
AVI 2024: International Conference on Advanced Visual Interfaces 2024
June 3 - 7, 2024
Arenzano, Genoa, Italy
Acceptance Rates
AVI '24 Paper Acceptance Rate 21 of 82 submissions, 26%;
Overall Acceptance Rate 128 of 490 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 34Total Downloads
- Downloads (Last 12 months)34
- Downloads (Last 6 weeks)6
Reflects downloads up to 26 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format