article

Multimodal embodied agents

Authors:

Catherine Pelachaud,

Isabella PoggiAuthors Info & Claims

The Knowledge Engineering Review, Volume 17, Issue 2

Pages 181 - 196

https://doi.org/10.1017/S0269888902000218

Published: 01 June 2002 Publication History

Abstract

1 Believable interactive embodied agentsAmong the goals of research on autonomous agents one important aim is to build believable interactive embodied agents that are apt to application to friendly interfaces in e-commerce, tourist and service query systems, entertainment (e.g. synthetic actors) and education (pedagogical agents, agents for help and instruction to the hearing impaired).

References

[1]

Andre, E, Rist, T, van Mulken, S, Klesen, M, and Baldes, S, 2000, The automated design of believable dialogues for animated presentation teams. In J. Cassell, J. Sullivan, S. P. and Churchill, E. (eds) Embodied Conversational Characters. MIT Press, Cambridge, MA.

[2]

Argyle, M, and Cook, M, 1976, Gaze and Mutual gaze. Cambridge University Press.

[3]

Ball, G, and Breese, J, 2000, Emotion and personality in a conversational agent. In J. Cassell, J. Sullivan, S. P. and Churchill, E. (eds) Embodied Conversational Characters. MIT Press, Cambridge, MA.

[4]

Beattie, G, 1981, Sequential temporal patterns of speech and gaze in dialogue. In Sebeok, T. and Umiker-Sebeok, J. (eds) Nonverbal Communication, Interaction, and Gesture, pages 297-320. The Hague, New-York.

[5]

Birdwhistell, R, 1952, Introduction to kinesics, an annotation system for analysis of body motion and gesture. University of Louisville.

[6]

Bok, S, 1982, Secrets. On the Ethics of Concealment and Revelation. Pantheon, New York.

[7]

Boyes-Braem, P and Braem, T, 1999, A pilot study of the expressive gestures used by classical orchestra conductors. In Emmorey, K. and H. Lane (eds) The signs of Language Revisited: An Anthology in Honor of Ursula Bellugi and Edward Klima. Laurence Erlbaum Associates.

[8]

Brown, P and Levinson, S, 1987, Politeness. Some Universals in Language Usage. Cambridge University Press, Cambridge.

[9]

Calbris, G, 1990, The semiotics of French gestures. University Press, Bloomington: Indiana.

[10]

Carbonell, J, 1980, Towards a process model of human personality traits. Artificial Intelligence, 15 49-74.

Digital Library

[11]

Carolis, ND, Pelachaud, C, Poggi, I, and de Rosis, F, 2001, Behavior planning for a reflexive agent. In IJCAI'01, Seattle, USA.

[12]

Cassell, J, 2000, Embodied conversational interface agents. Communications of the ACM, 43(4) 70-78.

Digital Library

[13]

Cassell, J, to appear, Embodied conversation: Integrating face and gesture into automatic spoken dialogue systems. In Luperfoy, editor, Spoken Dialogue Systems. MIT Press, Cambridge, MA.

[14]

Cassell, J, Bickmore, J, Billinghurst, M, Campbell, L, Chang, K, Vilhjálmsson, H, and Yan, H, 1999, Embodiment in conversational interfaces: Rea. In CHI'99, pages 520-527, Pittsburgh, PA.

[15]

Castelfranchi, C, de Rosis, F, Falcone, R, and Pizzutilo, S, 1997, A testbed for investigating personality-based multiagent cooperation. In European Summer School of Logic, Language and Information, Aix-en-Provence, France.

[16]

Castelfranchi, C, de Rosis, F, Falcone, R, and Pizzutilo, S, 1998, Personality traits and social attitudes in multi-agent cooperation". Applied Artificial Intelligence, 12(7-8) 649-675.

[17]

Castelfranchi, C and Poggi, I, 1998, Bugie finsioni sotterfugi. In Per una scienza dell'inganno. Carocci, Roma.

[18]

Chovil, N, 1991, Social determinants of facial displays. Journal of Nonverbal Behavior, 15(3) 141-154.

[19]

Condon, W, and Osgton, W, 1967, A segmentation of behavior. Journal of Psychiatric Research, 5 221-235.

[20]

deRosis, F and Grasso, F, 2000, Affective natural language generation. In Paiva, A. (ed) Affect in interactions. Springer-Verlag, Berlin.

[21]

Doenges, P, Lavagetto, F, Ostermann, J, Pandzic, I, and Petajan, E, 1997, MPEG-4: Audio/video and synthetic graphics/audio for mixed media. Image Communications Journal, 5(4).

[22]

Ducrot, O, 1984, Le dire et le dit. Les Editions de Minuit, Paris.

[23]

Duncan, S, 1974, Some signals and rules for taking speaking turns in conversations. In Weitz, S., editor, Nonverbal Communication. Oxford University Press.

[24]

Ekman, P, 1979, About brows: Emotional and conversational signals. In von Cranach, M., Foppa, K., Lepenies, W., and Ploog, D. (eds) Human ethology: Claims and limits of a new discipline: contributions to the Colloquium, pages 169-248. Cambridge University Press, Cambridge, England; New-York.

[25]

Ekman, P, 1982, Emotion in the human face. Cambridge University Press.

[26]

Ekman, P and Friesen, W, 1969, The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1.

[27]

Ekman, P and Friesen, W, 1978, Facial Action Coding System. Consulting Psychologists Press, Inc., Palo Alto, CA.

[28]

Elliott, C, 1992, An Affective Reasoner: A process model of emotions in a multiagent system. PhD thesis, Northwestern University, The Institute for the Learning Sciences. Technical Report No. 32.

[29]

Searle, J, 1992, (On) Searle on Conversation. John Benjamins Publishing Company.

[30]

Freedman, N, 1972, The analysis of movement behavior during the clinical interview. In Siegman, A. and Pope, B. (eds) Studies in Dyadic Communication, pages 177-210. Pergamon Press, New York.

[31]

Freedman, N and Grand, S, 1977, Communicative Structure and Psychic Structures. Plenum Press, New-York, London.

[32]

Fridlund, A, 1994, Human facial expression: An evolutionary view. Academic Press, New York.

[33]

Frijda, N and Swagerman, J, 1987, Can computers feel? Theory and design of an emotional system. Cognitive and Emotion, 1(3) 235-257.

[34]

Grice, HP, 1989, Studies in the Way of Words. Harvard University Press, Cambridge, MA.

[35]

Hall, R, 1966, The hidden dimension. Doubleday, New-York.

[36]

Ingenhoff, D and Schmitz, H, 2001 (in press), Com-trans: A multimedia tool for scientific transcriptions and analysis of communication. In M. Rector, I. Poggi, and N. Trigo (eds) Gestures, Meaning and Use. Universidad Fernando Pessoa, Porto.

[37]

Isbister, K and Nass, C, Forthcoming, Consistency of personality in interactive characters: Verbal cues, nonverbal cues, and user characteristics. International Journal of Human-Computer Studies.

[38]

Johnson, H., Ekman, P, and Friesen, W, 1975, Communicative body movements: American emblems. Semiotica, 15(4).

[39]

Johnson, W, Rickel, J, and Lester, J, 2000, Animated pedagogical agents: Face-to-face interaction in interactive learning environments. To appear in International Journal of Artificial Intelligence in Education.

[40]

Kendon, A, 1980, Gesticulation and speech: Two aspects of the process of utterance. In M.R. Key, editor, The Relation between Verbal and Nonverbal Communication, pages 207-227. Mouton.

[41]

Kendon, A, 1988a, How gestures can become like words. In Poyatos, F. (ed) Cross-cultural perspectives in nonverbal communication, pages 131-141. Hogrefe, Toronto.

[42]

Kendon, A, 1988b, Sign Languages of Aboriginal Australia: Cultural, semioti and communicative perspectives. Cambridge University Press, Cambridge.

[43]

Kendon, A, 1993, Human gesture. In Ingold, T. and Gibson, K. (eds) Tools, Language and Intelligence. Cambridge University Press, Cambridge.

[44]

Kendon, A, 1995, Gestures as illocutionary and discourse structure markers in southern Italian conversation. Journal of Pragmatics, 23 247-279.

[45]

Kendon, A, 1997a, Gesture. Annu. Rev. Anthropol., 26 109-128.

[46]

Kendon, A, 1997b, On gesture: Its complementary relationship with speech. In Siegman, A. and Feldstein, S. (eds) Nonverbal behavior and communication, pages 65-97. L. Erlbaum, Hillsdale, 2nd. edition edition.

[47]

Knapp, M, 1972, Nonverbal Communication in Human Interaction. Holt and Rinehart and Winston and Inc.

[48]

Kreidlin, G, 2001 (in press), The dictionary of Russian gestures. In C. Mueller and R. Posner (eds) The Semantics and Pragmatics of Everyday Gestures. Berlin Verlag Arno Spitz, Berlin.

[49]

Laban, R and Lawrence, F, 1974, Effort: Economy in body movement. Plays, Inc, Boston.

[50]

Lazarus, R, 1991, Emotion and adaptation. Oxford Press.

[51]

Lester, J, Stuart, S, Callaway, C, Voerman, J and Fitzgerald, P, 2000, Deictic and emotive communication in animated pedagogical agents. In J. Cassell, J. Sullivan, S. P. and Churchill, E. (eds) Embodied Conversational Characters. MIT Press, Cambridge, MA.

[52]

Magno-Caldognetto, E and Poggi, I, 1995, Creative iconic gestures: some evidence from aphasics. In Simone, R. (ed) Iconicity in Language, pages 257-275. John Benjamins, Amsterdam.

[53]

Martinho, C, Machado, I and Paiva, A, 2000, A cognitive approach to affective user modeling. In Paiva, A., editor, Affect in interactions. Springer-Verlag, Berlin.

[54]

McNeill, D, 1992, Hand and Mind: What Gestures Reveal about Thought. University of Chicago.

[55]

Mehrabian, A, 1969, Significance of posture and position in the communication of attitude and status relationships. Psychological Bulletin, 71(5).

[56]

Morris, D, 1977, Manwatching. Cape, London.

[57]

Morris, D, Collet, P, Marsh, P and O'Shaughnessy, M, 1979, Gestures. Their origins and distribution. Cape, London.

[58]

Ortony, A, Clore, G and Collins, A, 1988, The Cognitive Structure of Emotions. Cambridge University Press.

[59]

Paiva, A (ed) (2000, Affective Interactions. Towards a New Generation of Computer Interfaces. Springer, Berlin.

[60]

Payratò, L, 1993, A pragmatic view on autonomous gestures: A first repertoire of Catalan emblems. Journal of Pargmatics, 20 193-216.

[61]

Pennebaker, J, 1982, The psychology of physical symptoms. Springer-Verlag, New York.

[62]

Picard, R, 1997, Affective Computing. MIT Press, Cambridge.

[63]

Poggi, I, 2001, The italian gestionary. Meaning representation, ambiguity, and context. In C. Mueller and R. Posner (eds) The Semantics and Pragmatics of Everyday Gestures. Berlin Verlag Arno Spitz, Berlin.

[64]

Poggi, I and Mango-Caldognetto, E, 1996, A score for the analysis of gestures in multimodal communication. In L. Messing, editor, Proceedings of the Workshop on the Integration of Gesture and Language in Speech, Applied Science and Engineering Laboratories, pages 235-244, Newark and Wilmington, Del.

[65]

Poggi, I and Magno-Caldognetto, E, 1997, Mani che parlano. Gesti e Psicologia della comunicazione. Padova: Unipress.

[66]

Poggi, I, Cirella, F, and Zollo, A, In Prep., Touch as a communicative behavior.

[67]

Poggi, I and Mastropasqua, M, 1999, Multimodal communication and the conductor's face. In The Eighth International Workshop on the Cognitive Science of Natural Language Processing (CSNLP-8) "Language, Vision and Music", Galway, Ireland.

[68]

Poggi, I and Pelachaud, C, 1998, Performative faces. Speech Communication, 26:5-21.

Digital Library

[69]

Poggi, I and Pelachaud, C, 2000, Emotional meaning and expression in animated faces. In Paiva, A. (ed) Affect in interactions. Springer-Verlag, Berlin.

[70]

Poggi, I, Pezzato, N and Pelachaud, C, 1999, Gaze and its meaning in animated faces. In The Eighth International Workshop on the Cognitive Science of Natural Language Processing (CSNLP-8) "Language, Vision and Music", Galway, Ireland.

[71]

Posner, R and Serenari, M, 2001 (in press), The emergence of gestures from body movements. In Rector, M., Poggi, I. and Trigo, N. (eds) Gestures, Meaning and Use. Universidad Fernando Pessoa, Porto.

[72]

Prendinger, H and Ishizuka, M, 2001, Social role awareness in animated agents. In Proceedings of the 5th International Conference on Autonomous Agents, Montreal, Canada.

[73]

Rimé, B, 1987, Le partage social des emotions. In Rimé, B and Scherer, K (eds) Les emotions. Delachaux et Niestlé, Genève.

[74]

Rimé, B and Schiaratura, L, 1991, Gesture and speech. In Feldman, R and Rimé, B (eds) Fundamentals of Nonverbal Behavior. Cambridge University Press, Cambridge, New York.

[75]

Romagna, M, 1999, L'alfabeto dei gesti. parametri formazionali nel lessico dei gesti simbolici usati dagli udenti in italia. Tesi di laurea, Università Roma Tre.

[76]

Scheflen, A, 1973, Body Language and Social Order. Prentice-Hall, Inc.

[77]

Scherer, K, 1988, Facets of emotion: Recent research. Lawrence Erlbaum Associates Publishers.

[78]

Sowa, T and Wachsmuth, I, 2001 (in press), Coverbal iconic gestures for object dscriptions in virtual environments: an empirical study. In Rector, M, Poggi, I and Trigo, N (eds) Gestures, Meaning and Use. Universidad Fernando Pessoa, Porto.

[79]

Sparhawk, C, 1978, Identificational features of Persian gestures. Semiotica, 24 49-86.

[80]

Stokoe, W, 1978, Sign language structure: An outline of the communicative systems of the American deaf. Linstock Press, Silver Spring.

[81]

Tomkins, S, 1982, Affect, Imagery, Consciousness. Springer-Verlag, 1982.

[82]

Trappl, R and Petta, P (eds), 1997, Creating personalities for synthetic actors: Towards autonomous personality agents. Spinger.

[83]

Tumarkin, P, 2001 (in press), On a dictionary of Japanese gesture. In Rector, M, Poggi, I and Trigo, N (eds) Gestures, Meaning and Use. Universidad Fernando Pessoa, Porto.

[84]

Yan, H, 2000, Paired speech and gesture generation in embodied conversational agents. Master's thesis, MIT Media Laboratory.

Cited By

Chin SKim K(2018)Emotional intensity-based facial expression cloning for low polygonal applicationsIEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews10.1109/TSMCC.2008.201128339:3(315-330)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1109/TSMCC.2008.2011283
Hassani KNahvi AAhmadi A(2013)Architectural design and implementation of intelligent embodied conversational agents using fuzzy knowledge baseJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.5555/2596266.259629525:3(811-823)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.5555/2596266.2596295
Paggio PNavarretta C(2011)Head movements, facial expressions and feedback in Danish first encounters interactionsProceedings of the 6th international conference on Universal access in human-computer interaction: users diversity - Volume Part II10.5555/2027376.2027444(583-590)Online publication date: 9-Jul-2011
https://dl.acm.org/doi/10.5555/2027376.2027444

Multimodal embodied agents
1. Human-centered computing

Recommendations

Multimodal expressive embodied conversational agents
MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia

In this paper we present our work toward the creation of a multimodal expressive Embodied Conversational Agent (ECA). Our agent, called Greta, exhibits nonverbal behaviors synchronized with speech. We are using the taxonomy of communicative functions ...
Coalition formation and teamwork in embodied agents
Multimodal backchannels for embodied conversational agents
IVA'10: Proceedings of the 10th international conference on Intelligent virtual agents

One of the most desirable characteristics of an Embodied Conversational Agent (ECA) is the capability of interacting with users in a human-like manner. While listening to a user, an ECA should be able to provide backchannel signals through visual and ...

Comments

Information & Contributors

Information

Published In

cover image The Knowledge Engineering Review

The Knowledge Engineering Review Volume 17, Issue 2

June 2002

107 pages

ISSN:0269-8889

Issue’s Table of Contents

Publisher

Cambridge University Press

United States

Publication History

Published: 01 June 2002

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chin SKim K(2018)Emotional intensity-based facial expression cloning for low polygonal applicationsIEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews10.1109/TSMCC.2008.201128339:3(315-330)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1109/TSMCC.2008.2011283
Hassani KNahvi AAhmadi A(2013)Architectural design and implementation of intelligent embodied conversational agents using fuzzy knowledge baseJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.5555/2596266.259629525:3(811-823)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.5555/2596266.2596295
Paggio PNavarretta C(2011)Head movements, facial expressions and feedback in Danish first encounters interactionsProceedings of the 6th international conference on Universal access in human-computer interaction: users diversity - Volume Part II10.5555/2027376.2027444(583-590)Online publication date: 9-Jul-2011
https://dl.acm.org/doi/10.5555/2027376.2027444

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents