Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

The effect of avatar facial expressions on trust building in social virtual reality

Published: 20 October 2022 Publication History

Abstract

In immersive virtual reality (VR) applications, the facial expressions and lip-syncing of an avatar can have a significant impact on a user’s experience. In this paper, we designed a VR “trust game” scene to evaluate the effects of four expression conditions (positive facial expressions, neutral facial expressions, negative facial expressions and no expressions and lip-syncing) on participants in an immersive VR scene. We measured the participants with both objective and subjective measures. The two objective behavioral measures were the level of investment in the “trust game” and the users’ eye-movement data, and the subjective measures included social presence, emotional awareness level, and user preferences. We found that the participants were generally less trusting of the avatars with negative expressions, while the avatars with positive expressions made the participants feel comfortable and thus increased their willingness to cooperate with the avatars. In conclusion, avatars with facial expressions, whether positive or negative, were more effective in influencing the participants’ trust levels and decision-making behaviors than those without facial expressions. These findings provide novel ideas and suggestions for improving the level of human–computer interaction in VR and enhancing user experience in VR scenes.

References

[1]
Fernández-Dols JM and Russell JA The Psychology of Facial Expression 1997 Cambridge Cambridge University Press
[2]
Izard, C.E.: Emotions as motivations: an evolutionary-developmental perspective. In: Nebraska Symposium on Motivation, vol. 26, pp. 163–200 (1978). https://psycnet.apa.org/record/1982-11912-001
[3]
Ekman P and Oster H Facial expressions of emotion Annu. Rev. Psychol. 1979 30 1 527-554
[4]
Krach S, Hegel F, Wrede B, Sagerer G, Binkofski F, and Kircher T Can machines think? Interaction and perspective taking with robots investigated via FMRI PLoS ONE 2008 3 7 1-11
[5]
Berg J, Dickhaut J, and McCabe K Trust, reciprocity, and social history Games Econom. Behav. 1995 10 1 122-142
[6]
Scharlemann JP, Eckel CC, Kacelnik A, and Wilson RK The value of a smile: game theory with a human face J. Econ. Psychol. 2001 22 5 617-640
[7]
Krumhuber E, Manstead AS, Cosker D, Marshall D, Rosin PL, and Kappas A Facial dynamics as indicators of trustworthiness and cooperative behavior Emotion 2007 7 4 730
[8]
Agarwal S, Santra B, and Mukherjee DP Anubhav: recognizing emotions through facial expression Vis. Comput. 2018 34 2 177-191
[9]
Smith, H.J., Neff, M.: Communication behavior in embodied virtual reality. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2018).
[10]
Pan Y and Steed A The impact of self-avatars on trust and collaboration in shared virtual environments PLoS ONE 2017 12 12 0189078
[11]
Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., Sasse, M.A.: The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 529–536 (2003).
[12]
Heidicker, P., Langbehn, E., Steinicke, F.: Influence of avatar appearance on presence in social VR. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp. 233–234 (2017). IEEE
[13]
Yoon, B., Kim, H.-I., Lee, G.A., Billinghurst, M., Woo, W.: The effect of avatar appearance on social presence in an augmented reality remote collaboration. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 547–556. IEEE (2019).
[14]
Cho, S., Kim, S.-W., Lee, J., Ahn, J., Han, J.: Effects of volumetric capture avatars on social presence in immersive virtual environments. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 26–34. IEEE (2020).
[15]
Aseeri S and Interrante V The influence of avatar representation on interpersonal communication in virtual social environments IEEE Trans. Vis. Comput. Graph. 2021 27 5 2608-2617
[16]
Latoschik, M.E., Roth, D., Gall, D., Achenbach, J., Waltemate, T., Botsch, M.: The effect of avatar realism in immersive social virtual realities. In: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, pp. 1–10 (2017).
[17]
Zibrek K, Kokkinara E, and McDonnell R The effect of realistic appearance of virtual characters in immersive environments-does the character’s personality play a role? IEEE Trans. Vis. Comput. Graph. 2018 24 4 1681-1690
[18]
Gonzalez-Franco M, Steed A, Hoogendyk S, and Ofek E Using facial animation to increase the enfacement illusion and avatar self-identification IEEE Trans. Vis. Comput. Graph. 2020 26 5 2023-2029
[19]
Hart, J.D., Piumsomboon, T., Lawrence, L., Lee, G.A., Smith, R.T., Billinghurst, M.: Emotion sharing and augmentation in cooperative virtual reality games. In: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, pp. 453–460 (2018).
[20]
Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., Debevec, P.: Multiview face capture using polarized spherical gradient illumination. In: Proceedings of the 2011 SIGGRAPH Asia Conference, pp. 1–10 (2011).
[21]
Kampouris, C., Zafeiriou, S., Ghosh, A.: Diffuse-specular separation using binary spherical gradient illumination. In: Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations. SR’18, pp. 1–10. Eurographics Association, Goslar, DEU (2018).
[22]
Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH’00, pp. 145–156. ACM Press/Addison-Wesley Publishing Co., USA (2000).
[23]
Debevec P, Wenger A, Tchou C, Gardner A, Waese J, and Hawkins T A lighting reproduction approach to live-action compositing ACM Trans. Graph. 2002 21 3 547-556
[24]
Wenger A, Gardner A, Tchou C, Unger J, Hawkins T, and Debevec P Performance relighting and reflectance transformation with time-multiplexed illumination ACM Trans. Graph. 2005 24 3 756-764
[25]
Alexander O, Rogers M, Lambeth W, Chiang J-Y, Ma W-C, Wang C-C, and Debevec P The digital Emily project: achieving a photorealistic digital actor IEEE Comput. Graph. Appl. 2010 30 4 20-31
[27]
Taylor S, Kim T, Yue Y, Mahler M, Krahe J, Rodriguez AG, Hodgins J, and Matthews I A deep learning approach for generalized speech animation ACM Trans. Graph. (TOG) 2017 36 4 1-11
[28]
Karras T, Aila T, Laine S, Herva A, and Lehtinen J Audio-driven facial animation by joint end-to-end learning of pose and emotion ACM Trans. Graph. (TOG) 2017 36 4 1-12
[29]
Olszewski K, Lim JJ, Saito S, and Li H High-fidelity facial and speech animation for VR HMDS ACM Trans. Graph. (TOG) 2016 35 6 1-14
[30]
Burgos-Artizzu, X.P., Fleureau, J., Dumas, O., Tapie, T., LeClerc, F., Mollet, N.: Real-time expression-sensitive HMD face reconstruction. In: SIGGRAPH Asia 2015 Technical Briefs, pp. 1–4 (2015).
[31]
Guo K, Xu F, Yu T, Liu X, Dai Q, and Liu Y Real-time geometry, albedo, and motion reconstruction using a single RGB-D camera ACM Trans. Graph. (ToG) 2017 36 4 1
[32]
Wei S-E, Saragih J, Simon T, Harley AW, Lombardi S, Perdoch M, Hypes A, Wang D, Badino H, and Sheikh Y VR facial animation via multiview image translation ACM Trans. Graph. (TOG) 2019 38 4 1-16
[33]
Suzuki, K., Nakamura, F., Otsuka, J., Masai, K., Itoh, Y., Sugiura, Y., Sugimoto, M.: Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display. In: 2017 IEEE Virtual Reality (VR), pp. 177–185. IEEE (2017).
[34]
Tortosa, M.I., Strizhko, T., Capizzi, M., Ruz, M.: Interpersonal effects of emotion in a multi-round trust game. Psicologica Int. J. Methodol. Exp. Psychol. 34(2), 179–198 (2013). https://eric.ed.gov/?id=EJ1018812
[35]
Stouten J and De Cremer D Seeing is believing: the effects of facial expressions of emotion and verbal communication in social dilemmas J. Behav. Decis. Mak. 2010 23 3 271-287
[36]
Jaeger B, Evans AM, Stel M, and van Beest I Explaining the persistent influence of facial cues in social decision-making J. Exp. Psychol. Gen. 2019 148 6 1008
[37]
Mussel, P., Göritz, A.S., Hewig, J.: The value of a smile: facial expression affects ultimatum-game responses. Judgm. Decis. Mak. 8(3) (2013). https://nbn-resolving.org/urn:nbn:de:bvb:20-opus-129639
[38]
Mathur, M.B., Reichling, D.B.: An uncanny game of trust: social trustworthiness of robots inferred from subtle anthropomorphic facial cues. In: 2009 4th ACM/IEEE International Conference on Human–Robot Interaction (HRI), pp. 313–314 (2009).
[39]
Wu, J., Paeng, E., Linder, K., Valdesolo, P., Boerkoel, J.: Trust and cooperation in human–robot decision making. https://www.aaai.org/ocs/index.php/FSS/FSS16/paper/view/14118/13675 (2016)
[40]
Paradeda RB, Hashemian M, Rodrigues RA, and Paiva A Agah A, Cabibihan J-J, Howard AM, Salichs MA, and He H How facial expressions and small talk may influence trust in a robot Social Robotics 2016 Cham Springer 169-178
[41]
George, C., Eiband, M., Hufnagel, M., Hussmann, H.: Trusting strangers in immersive virtual reality. In: Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion. IUI’18 Companion. Association for Computing Machinery, New York, NY, USA (2018).
[42]
Kugler T, Ye B, Motro D, and Noussair CN On trust and disgust: evidence from face reading and virtual reality Soc. Psychol. Pers. Sci. 2020 11 3 317-325
[43]
Harjunen V, Spape M, Ahmed I, Jacucci G, and Ravaja N Persuaded by the machine: the effect of virtual nonverbal cues and individual differences on compliance in economic bargaining Comput. Hum. Behav. 2018 87 11
[44]
Mori M, MacDorman KF, and Kageki N The uncanny valley [from the field] IEEE Robotics Autom. Mag. 2012 19 2 98-100
[45]
Ekman, P., Friesen, W.V.: Facial Action Coding System. American Psychological Association (APA) (1978).
[46]
Meneses, J.A.C., Díaz, J.M.M.: Vocal emotion expressions effects on cooperation behavior. Psicológica 38(1), 1–24 (2017). https://www.redalyc.org/pdf/169/16949050001.pdf
[48]
Harms, P.C., Biocca, P.F.: Internal Consistency and Reliability of the Networked Minds Measure of Social Presence. http://cogprints.org/7026/ (2004)
[49]
Biocca, P.F., Harms, P.C.: Guide to the Networked Minds Social Presence Inventory v. 1.2. http://cogprints.org/6743/ (2003)
[50]
Boyle GJ Reliability and validity of Izard’s differential emotions scale Pers. Individ. Differ. 1984 7 747-750
[51]
Tu, Z., Weng, D., Cheng, D., Shen, R., Fang, H., Bao, Y.: The effect of two different types of human–computer interactions on user’s emotion in virtual counseling environment. In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 217–221 (2019).
[52]
Pauw, L.S., Sauter, D.A., van Kleef, G.A., Lucas, G.M., Gratch, J., Fischer, A.H.: The avatar will see you now: support from a virtual human provides socio-emotional benefits. Comput. Hum. Behav. 136, 107368 (2022).
[53]
Shao, D., Lee, I.-J.: Acceptance and influencing factors of social virtual reality in the urban elderly. Sustainability 12(22) (2020).
[54]
Baker, S., Kelly, R.M., Waycott, J., Carrasco, R., Hoang, T., Batchelor, F., Ozanne, E., Dow, B., Warburton, J., Vetere, F.: Interrogating social virtual reality as a communication medium for older adults. Proc. ACM Hum. Comput. Interact. 3(CSCW) (2019).

Cited By

View all
  • (2024)Exploring the Impact of Non-Verbal Virtual Agent Behavior on User Engagement in Argumentative DialoguesProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688315(224-232)Online publication date: 24-Nov-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image The Visual Computer: International Journal of Computer Graphics
The Visual Computer: International Journal of Computer Graphics  Volume 39, Issue 11
Nov 2023
761 pages

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 20 October 2022
Accepted: 04 October 2022

Author Tags

  1. Virtual reality
  2. Face expression
  3. Trust game
  4. Avatar

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring the Impact of Non-Verbal Virtual Agent Behavior on User Engagement in Argumentative DialoguesProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688315(224-232)Online publication date: 24-Nov-2024

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media