Animated Agents’ Facial Emotions: Does the Agent Design Make a Difference?
Pages 10 - 25
Abstract
The paper reports ongoing research toward the design of multimodal affective pedagogical agents that are effective for different types of learners and applications. In particular, the work reported in the paper investigated the extent to which the type of character design (realistic versus stylized) affects students’ perception of an animated agent’s facial emotions, and whether the effects are moderated by learner characteristics (e.g. gender). Eighty-two participants viewed 10 animation clips featuring a stylized character exhibiting 5 different emotions, e.g. happiness, sadness, fear, surprise and anger (2 clips per emotion), and 10 clips featuring a realistic character portraying the same emotional states. The participants were asked to name the emotions and rate their sincerity, intensity, and typicality. The results indicated that for recognition, participants were slightly more likely to recognize the emotions displayed by the stylized agent, although the difference was not statistically significant. The stylized agent was on average rated significantly higher for facial emotion intensity, whereas the differences in ratings for typicality and sincerity across all emotions were not statistically significant. A significant difference in ratings was shown in regard to sadness (within typicality), happiness (within sincerity), fear, anger, sadness and happiness (within intensity) with the stylized agent rated higher. Gender was not a significant correlate across all emotions or for individual emotions.
References
[1]
Adamo-Villani, N., Beni, G.: Keyboard encoding of facial expressions. In: 8th International Conference on Information Visualization-HCI Symposium (IV04) Proceedings, pp. 324–328. IEEE, London (2004)
[2]
Adamo-Villani N and Wilbur R Two novel technologies for accessible Math and Science Education IEEE Multimedia Spec. Issue Access. 2008 15 38-46
[3]
Adamo-Villani N, Lestina J, and Anasingaraju S Vincenti G, Bucciero A, and Vaz de Carvalho C Does character’s visual style affect viewer’s perception of signing avatars? E-Learning, E-Education, and Online Training 2016 Cham Springer 1-8
[4]
Adolphs R Perception and emotion: How we recognize facial expressions Current Dir. Psychol. Sci. 2006 15 5 222-226
[5]
Albrecht I, Schroeder M, Haber J, and Seidel H-S Mixed feelings: expression of non-basic emotions in a muscle-based talking head Virtual Real. (special issue) 2005 8 201-212
[6]
Bancroft T Creating Characters with Personality 2016 New York Watson-Guptil
[7]
Baylor A and Kim S Designing nonverbal communication for pedagogical agents: when less is more Comput. Human Behav. 2009 25 2 450-457
[8]
Becker, C., Wachsmuth, I.: Modeling primary and secondary emotions for a believable communication agent. In: Proceedings of International Workshop on Emotion and Computing, in Conjunction with the 29th annual German Conference on Artificial Intelligence (KI2006), Bremen, Germany, pp. 31–34 (2006)
[9]
Calder A, Rhodes G, Johnson M, and Haxby J Oxford Handbook of Face Perception 2011 Oxford Oxford University Press
[10]
Cissel, K.: A study of the effects of computer animated character body style on perception of facial expression. Purdue University e-Pub, MS thesis (2015). https://docs.lib.purdue.edu/cgttheses/28/
[11]
Courgeon, M., Buisine, S., Martin, J.C.: Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. In: Proceedings of 9th International Conference, IVA 2009, Netherlands, pp. 201–214 (2009)
[12]
D’Mello S and Graesser A Durlach PJ and Lesgold AM Emotions during learning with AutoTutor Adaptive Technologies for Training and Education 2012 New York Cambridge University Press 117-139
[13]
D’Mello S and Graesser A AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back ACM Transact. Interact. Intell. Syst. 2012 2 4 23
[14]
Davis, C., Kim, J., Kuratate, T., Burnham, D.: Making a thinking-talking head. In: Proceedings of the International Conference on Auditory-Visual Speech Processing (AVSP 2007), Hilvarenbeek. The Netherlands (2007)
[15]
Domagk S Do pedagogical agents facilitate learner motivation and learning outcomes? The role of the appeal of agent’s appearance and voice J. Media Psychol. 2010 22 2 84-97
[16]
Ekman P and Friesen VW Unmasking the Face. A guide to Recognizing Emotions from Facial Expressions 1975 Cambridge Malor Books
[17]
Ekman, P., Friesen, V.W.: Manual for the Facial Action Coding System. Consulting Psychologists Press (1977)
[18]
Garcia-Rojas A et al. Emotional face expression profiles supported by virtual human ontology Comp. Anim. Virtual Worlds 2006 17 259-269
[19]
Guo YR and Goh HL Affect in embodied pedagogical agents: meta-analytic review J. Educat. Comput. Res. 2015 53 1 124-149
[20]
Holmes J Designing agents to support learning by explaining Comput. Educ. 2007 48 523-547
[21]
Huang, T.S., Cohen, I., Hong, P., Li, Y.: IA3: Intelligent Affective Animated Agents. In: Proceedings of ICIG 2000 (2000)
[22]
Hyde, J., Carter, E., Kiesler, S., Hodgins, J.K.: Perceptual effects of damped and exaggerated facial motion in animated characters. In: Proceedings of the 10th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2013) (2013)
[23]
Intense. (n.d.). In Merriam-Webster. https://www.merriam-webster.com/dictionary/intense
[24]
Johnson WL and Rickel J Steve: an animated pedagogical agent for procedural training in virtual environments SIGART Bull. 1998 8 16-21
[25]
Kim S and Baylor A Pedagogical agents as social models to influence learner attitudes Educat. Technol. 2007 47 01 23-28
[26]
Kim Y and Lim J Gendered socialization with an embodied agent: creating a social and affable mathematics learning environment for middle-grade females J. Educat. Psychol. 2013 105 4 1164-1174
[27]
Kim Y and Wei Q The impact of user attributes and user choice in an agent-based environment Comput. Educat. 2011 56 505-514
[28]
Kim Y, Baylor A, and Shen E Pedagogical agents as learning companions: the impact of agent emotion and gender J. Comput. Assist. Learn. 2007 23 220-234
[29]
Lester, J., Voerman, J., Towns, S.J., Callaway, C.: Cosmo: a life-like animated pedagogical agent with deictic believability. In: Notes of the IJCAI 1997 Workshop on Animated Interface Agents: Making Them Intelligent, Nagoya, Japan, pp. 61–70 (1997)
[30]
Lester J, Stone B, and Stelling G Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments User Model. User-Adap. Inter. 1999 9 1–2 1-44
[31]
Lester J, Towns SG, and Fitzgerald P Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents Int. J. Artif. Intell. Educ. 1999 10 3–4 278-291
[32]
Lester, J., Converse, S., Kahler, S., Barlow, T., Stone, B., Bhogal, R.: The persona effect: affective impact of animated pedagogical agents. In: Proceedings of CHI 1997, pp. 359–366 (1997)
[33]
Lester, J., Converse, S., Stone, B., Kahler, S., Barlow, T.: Animated pedagogical agents and problem-solving effectiveness: a large-scale empirical evaluation. In: Proceedings of the 8th World Conference on Artificial Intelligence in Education, pp. 23–30 (1997)
[34]
Lisetti, C., Nasoz, F.: MAUI: a multimodal affective user interface. In: Proceedings 10th ACM Int’l Conf. Multimedia (Multimedia 2002), pp. 161–170 (2002)
[35]
Martinez, A., Benavente. R.: The AR Face Database. CVC Technical Report #24 (1998)
[36]
Mayer, R.E.: Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In Mayer, R.E. (ed.) The Cambridge handbook of multimedia learning, 2nd ed, pp. 345–368. Cambridge University Press, New York (2014)
[37]
Mayer RE and DaPra S An embodiment effect in computer-based learning with animated pedagogical agent J. Exp. Psychol. Appl. 2012 18 239-252
[38]
McCloud S Understanding Comics: The Invisible Art 1993 Mass Northampton
[39]
McDonnell, R., Jorg, S., JMcHugh, J., Newell, F., O’Sullivan, C.: Evaluating the emotional content of human motions on real and virtual characters. In: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, pp. 67–73 (2008)
[40]
Morency, L.P., et al.: Demonstration: a perceptive virtual human interviewer for healthcare applications. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
[41]
Moreno R and Flowerday T Students’ choice of animated pedagogical agents in science learning: a test of the similarity attraction hypothesis on gender and ethnicity Contemp. Educ. Psychol. 2006 31 186-207
[42]
Moreno R and Mayer RE Interactive Multimodal Learning Environments Educ. Psychol. Rev. 2007 19 309-326
[43]
Nass, C., Isbister, K., Lee, E.J.: Truth is beauty: researching embodied conversational agents. Embodied Conversational Agents, pp. 374–402 (2000)
[44]
Paleari, M., Lisetti, C.: Psychologically grounded avatars expressions. In: Proceedings of First Workshop on Emotion and Computing at KI 2006, 29th Annual Conf. on Artificial Intelligence, Bremen, Germany (2006)
[45]
Pandzic IS and Forcheimer R MPEG4 facial animation – The standard, implementations and applications 2002 Chichester John Wiley, Sons
[46]
Pelachaud C Modelling multimodal expression of emotion in a virtual agent Phil. Trans. R. Soc. B 2009 364 3539-3548
[47]
Powers, D., Leibbrandt, R., Luerssen, M., Lewis, T., Lawson, M.: PETA-a pedagogical embodied teaching agent. In: Proceedings of PETRA08: 1st international conference on PErvasive Technologies Related to Assistive Environments, Athens (2008)
[48]
Riek, D.L., Rabinowitch, T.C., Chakrabarti, B., Robinson, P.: How anthropomorphism affects empathy toward robots. In: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pp. 245–246 (2009)
[49]
Rosenberg-Kima RB, Baylor A, Plant A, and Doerr C Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs Comput. Hum. Behav. 2008 24 2741-2756
[50]
Ruttkay Z, Dormann C, and Noot H Ruttkay Z and Pelachaud C Embodied conversational agents on a common ground From Brows to Trust 2004 Dordrecht Springer 27-66
[51]
Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Scherer, K., Schorr, A., Johnstone, T. (eds.) Appraisal processes in emotion: theory, methods, research, pp. 92–119. Oxford University Press, New York (2001)
[52]
Schroeder NL, Adesope O, and Barouch Gilbert R How effective are pedagogical agents for learning? A Meta-Analytic Review J. Educ.Comput. Res. 2013 49 1 1-39
[53]
Sincere. (n.d). In Oxford Dictionaries Online. https://en.oxforddictionaries.com/definition/sincere
[54]
Thomas, F., Johnston, O.: The Illusion of Life. Disney Animation. Disney Editions, Los Angeles (1995)
[55]
Typical. (n.d.). In Oxford Dictionaries online. https://en.oxforddictionaries.com/definition/typical
[56]
Wallraven C, Breidt M, Cunningham DW, and Bulthoff HH Evaluating the perceptual realism of animated facial expressions ACM Transact. Appl. Percept. 2007 23 1-20 http://dl.acm.org/citation.cfm?id=127876456
[57]
Wang N, Johnson L, Mayer RE, Rizzo P, Shaw E, and Collins H The politeness effect: pedagogical agents and learning outcomes Int. J. Hum. Comput. Stud. 2008 66 98-112
[58]
Welch R, Blackmon T, Liu A, and Stark L The effects of pictorial realism, delay of visual feedback, and observer interactivity on the subjective sense of presence Presence Teleoperators Virtual Environ. 1996 5 263-273
[59]
Willcox G The feeling wheel: a tool for expanding awareness of emotions and increasing spontaneity and intimacy Transact. Anal. J. 1982 12 4 274-276
[60]
Woo HL Designing multimedia learning environments using animated pedagogical agents: factors and issues J. Comput. Assist. Learn. 2009 25 3 203-218
[61]
Zhou L, Mohammed A, and Zhang D Mobile personal information management agent: Supporting natural language interface and application integration Inf. Process. Manage. 2012 48 1 23-31
Recommendations
Generating Facial Emotions for Diagnosis and Training
CBMS '15: Proceedings of the 2015 IEEE 28th International Symposium on Computer-Based Medical SystemsThe ability to process and identify facial emotions is an essential factor for an individuals social interaction. There are certain psychiatric disorders that can limit an individuals ability to recognize emotions in facial expressions. This problem ...
Bodily Expression of Emotions in Animated Agents
Advances in Visual ComputingAbstractThe goal of this research is to identify key affective body gestures that can clearly convey four emotions, namely happy, content, bored, and frustrated, in animated characters that lack facial features. Two studies were conducted, a first to ...
INFLUENCE OF AUTONOMIC SIGNALS ON PERCEPTION OF EMOTIONS IN EMBODIED AGENTS
Intelligent Virtual AgentsSpecific patterns of autonomic activity have been reported when people experience emotions. Typical autonomic signals that change with emotion are wrinkles, blushing, sweating, tearing, and respiration. This article explores whether these signals can ...
Comments
Information & Contributors
Information
Published In
Jun 2019
427 pages
© Springer Nature Switzerland AG 2019.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 28 July 2019
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in