Abstract
With the advent of new technologies in everyday life, leisure, or therapy, we will increasingly interact with a non-human virtual character. Understanding facial expressions and intentions of these virtual agents is important to enable them to achieve their goals. The objective of our study is to assess whether expressions are perceived as being positive or negative on faces more or less similar to those of humans. Eighty-three undergraduate students took part in a computerized emotion recognition task. The participants had to identify whether each face expressed a positive or a negative emotion. Eight different faces (human, avatar, mesh, and robot) were shown 38 times each on a computer screen. Each face was represented by a photo. Response time and the number of correct responses were recorded. Our research has raised important points: the accuracy and time taken for emotion recognition were found to be similar on human or avatar faces. On the other hand, as soon as these faces were too ambiguous or schematic, emotion recognition capacities were found to be diminished.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adolphs, R. (1999). Social cognition and the human brain. Trends in Cognitive Sciences, 3(12), 11.
Adolphs, R., Baron-Cohen, S., & Tranel, D. (2002). Impaired recognition of social emotions following amygdala damage. Journal of Cognitive Neuroscience, 14(8), 1264–1274. https://doi.org/10.1162/089892902760807258.
Alkawaz, M. H., & Basori, A. H. (2012). The effect of emotional colour on creating realistic expression of avatar. Proceedings of the 11th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in industry - VRCAI ‘12, 143. https://doi.org/10.1145/2407516.2407555.
Bailenson, J. N., Blascovich, J., Beall, A. C., & Loomis, J. M. (2003). Interpersonal distance in immersive virtual environments. Personality and Social Psychology Bulletin, 29(7), 819–833. https://doi.org/10.1177/0146167203029007002.
Beck, A. T., Steer, R. A., Ball, R., & Ranieri, W. F. (1996). Comparison of Beck depression inventories-IA and-II in psychiatric outpatients. Journal of Personality Assessment, 67(3), 588–597. https://doi.org/10.1207/s15327752jpa6703_13.
Biocca, F., Harms, C., & Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: Review and suggested criteria. In Presence: Teleoperators and Virtual Environments, 456–480.
Breazeal, C. (2003). Emotion and sociable humanoid robots. International Journal of Human-Computer Studies, 59(1–2), 119–155. https://doi.org/10.1016/S1071-5819(03)00018-1.
Cheetham, M., Wu, L., Pauli, P., & Jancke, L. (2015). Arousal, valence, and the uncanny valley: Psychophysiological and self-report findings. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.00981.
Churamani, N., Cruz, F., Griffiths, S., & Barros, P. (2016). iCub: Learning emotion expressions using human reward.
Dubal, S., Foucher, A., Jouvent, R., & Nadel, J. (2011). Human brain spots emotion in non humanoid robots. Social Cognitive and Affective Neuroscience, 6(1), 90–97. https://doi.org/10.1093/scan/nsq019.
Dyck, M., Winbeck, M., Leiberg, S., Chen, Y., Gur, R. C., & Mathiak, K. (2008). Recognition profile of emotions in natural and virtual faces. PLoS One, 3(11), e3628. https://doi.org/10.1371/journal.pone.0003628.
Dyck, M., Winbeck, M., Leiberg, S., Chen, Y., & Mathiak, K. (2010). Virtual faces as a tool to study emotion recognition deficits in schizophrenia. Psychiatry Research, 179(3), 247–252. https://doi.org/10.1016/j.psychres.2009.11.004.
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384–392. https://doi.org/10.1037/0003-066X.48.4.384.
Eyssel, F., Hegel, F., Horstmann, G., & Wagner, C. (2010). Anthropomorphic inferences from emotional nonverbal cues: A case study. 19th International Symposium in Robot and Human Interactive Communication, 646-651. https://doi.org/10.1109/ROMAN.2010.5598687.
Frith, C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3453–3458. https://doi.org/10.1098/rstb.2009.0142.
Garau, M., Friedman, D., Widenfeld, H. R., Antley, A., Brogni, A., & Slater, M. (2008). Temporal and Spatial Variations in Presence: Qualitative Analysis of Interviews from an Experiment on Breaks in Presence. Presence: Teleoperators and Virtual Environments, 17(3), 293–309. https://doi.org/10.1162/pres.17.3.293
Hall, J., Tritton, T., Rowe, A., Pipe, A., Melhuish, C., & Leonards, U. (2014). Perception of own and robot engagement in human–robot interactions and their dependence on robotics knowledge. Robotics and Autonomous Systems, 62(3), 392–399. https://doi.org/10.1016/j.robot.2013.09.012.
Hortensius, R., Hekele, F., & Cross, E. S. (2018). The perception of emotion in artificial agents. IEEE Transactions on Cognitive and Developmental Systems, 10(4), 852–864. https://doi.org/10.1109/TCDS.2018.2826921.
Jack, R. E., & Schyns, P. G. (2017). Toward a social psychophysics of face communication. Annual Review of Psychology, 68(1), 269–297. https://doi.org/10.1146/annurev-psych-010416-044242.
Joyal, C. C., Jacob, L., Cigna, M.-H., Guay, J.-P., & Renaud, P. (2014). Virtual faces expressing emotions: An initial concomitant and construct validity study. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00787.
Kahn Jr., P. H., Kanda, T., Ishiguro, H., Freier, N. G., Severson, R. L., Gill, B. T., Ruckert, J. H., & Shen, S. (2012). “Robovie, you’ll have to go into the closet now”: Children’s social and moral relationships with a humanoid robot. Developmental Psychology, 48(2), 303–314. https://doi.org/10.1037/a0027033.
Lawrence, K., Campbell, R., & Skuse, D. (2015). Age, gender, and puberty influence the development of facial emotion recognition. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.00761.
Lazzeri, N., Mazzei, D., Greco, A., Rotesi, A., Lanatà, A., & De Rossi, D. E. (2015). Can a humanoid face be expressive? A psychophysiological investigation. Frontiers in Bioengineering and Biotechnology, 3. https://doi.org/10.3389/fbioe.2015.00064.
Leppänen, J. M., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research Psychologische Forschung, 69(1–2), 22–29. https://doi.org/10.1007/s00426-003-0157-2.
Liebowitz, M. R. (1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, 141–173. https://doi.org/10.1159/000414022.
MacDorman, K. F. (2019). La Vallée de l’Étrange de Mori Masahiro: Importance et impact sur l’esthétique et la conception des robots. e-Phaïstos, VII-2. https://doi.org/10.4000/ephaistos.5333.
Milcent, A.-S., Geslin, E., Kadri, A., & Richir, S. (2019). Expressive virtual human: Impact of expressive wrinkles and pupillary size on emotion recognition. Proceedings of the 19th ACM international conference on intelligent virtual agents, 215–217. https://doi.org/10.1145/3308532.3329446.
Mill, A., Allik, J., Realo, A., & Valk, R. (2009). Age-related differences in emotion recognition ability: A cross-sectional study. Emotion, 9(5), 619–630. https://doi.org/10.1037/a0016562.
Mori, M. (1970). Bukimi no tani [the uncanny valley]. Energy, 7, 33–35.
Moser, E., Derntl, B., Robinson, S., Fink, B., Gur, R. C., & Grammer, K. (2007). Amygdala activation at 3T in response to human and avatar facial expressions of emotions. Journal of Neuroscience Methods, 161(1), 126–133. https://doi.org/10.1016/j.jneumeth.2006.10.016.
Nummenmaa, L., & Calvo, M. G. (2015). Dissociation between recognition and detection advantage for facial expressions: A meta-analysis. Emotion, 15(2), 243–256. https://doi.org/10.1037/emo0000042.
Oberman, L. M., Winkielman, P., & Ramachandran, V. S. (2007). Face to face: Blocking facial mimicry can selectively impair recognition of emotional expressions. Social Neuroscience, 2(3–4), 167–178. https://doi.org/10.1080/17470910701391943.
Oztop, E., Franklin, D. W., Chaminade, T., & Cheng, G. (2005). Human–humanoid interaction: Is a humanoid robot perceived as a human? International Journal of Humanoid Robotics, 537–559.
Pais, A. L., Argall, B. D., & Billard, A. G. (2013). Assessing interaction dynamics in the context of robot programming by demonstration. International Journal of Social Robotics, 5(4), 477–490. https://doi.org/10.1007/s12369-013-0204-0.
Pertaub, D.-P., Slater, M., & Barker, C. (2001). An experiment on fear of public speaking in virtual reality: (705412011-025) [data set]. American Psychological Association. https://doi.org/10.1037/e705412011-025.
Raffard, S., Bortolon, C., Khoramshahi, M., Salesse, R. N., Burca, M., Marin, L., Bardy, B. G., Billard, A., Macioce, V., & Capdevielle, D. (2016). Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study. Schizophrenia Research, 176(2), 506–513. https://doi.org/10.1016/j.schres.2016.06.001.
Seyama, J., & Nagayama, R. (2007). The uncanny valley: Effect of realism on the impression of artificial human faces. Presence, 16, 337–351. https://doi.org/10.1162/pres.16.4.337.
Sjölie, D. (2012). Presence and general principles of brain function. Interacting with Computers, 24(4 SI), 193–202.
Spencer-Smith, J., Wild, H., Innes-Ker, Å. H., Townsend, J., Duffy, C., Edwards, C., Ervin, K., Merritt, N., & Pair, J. W. (2001). Making faces: Creating three-dimensional parameterized models of facial expression. Behavior Research Methods, Instruments, & Computers, 33(2), 115–123. https://doi.org/10.3758/BF03195356.
Spielberger, C. D. (1983). Manual for the state-trait anxiety inventory STAI (Form Y) (« Self-Evaluation Questionnaire »). http://ubir.buffalo.edu/xmlui/handle/10477/1873
Todorov, A., Olivola, C. Y., Dotsch, R., & Mende-Siedlecki, P. (2015). Social attributions from faces: Determinants, consequences, accuracy, and functional significance. Annual Review of Psychology, 66(1), 519–545. https://doi.org/10.1146/annurev-psych-113011-143831.
Wendt, C. S., & Berg, G. (2009). Nonverbal humor as a new dimension of HRI. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, 183–188. https://doi.org/10.1109/ROMAN.2009.5326230.
Zajonc, R. B. (1998). Emotions. In The handbook of social psychology, Vols. 1–2, 4th ed (p. 591–632). McGraw-Hill.
Acknowledgments
Thanks to Vrutti Joshi, psychologist, for her proofreading and help for writing process.
Funding
We are grateful to the Provence-Alpes-Côte d’Azur Regional Council for his financial support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cerda, L., Graziani, P. & Del-Monte, J. Emotional Valence Recognition on Virtual, Robotic, and Human Faces: a Comparative Study. J. technol. behav. sci. 6, 151–158 (2021). https://doi.org/10.1007/s41347-020-00172-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41347-020-00172-5