Article

Free access

Non-verbal cues for discourse structure

Authors:

Justine Cassell,

Yukiko I. Nakano,

Timothy W. Bickmore,

Candace L. Sidner,

Charles RichAuthors Info & Claims

ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics

Pages 114 - 123

https://doi.org/10.3115/1073012.1073028

Published: 06 July 2001 Publication History

Abstract

This paper addresses the issue of designing embodied conversational agents that exhibit appropriate posture shifts during dialogues with human users. Previous research has noted the importance of hand gestures, eye gaze and head nods in conversations between embodied agents and humans. We present an analysis of human monologues and dialogues that suggests that postural shifts can be predicted as a function of discourse state in monologues, and discourse and conversation state in dialogues. On the basis of these findings, we have implemented an embodied conversational agent that uses Collagen in such a way as to generate postural shifts.

References

[1]

Andre, E., Rist, T., & Muller, J., Employing Al methods to control the behavior of animated interface agents, Applied Artificial Intelligence, vol. 13, pp. 415--448, 1999.

[2]

Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H., & Yan, H., Embodiment in Conversational Interfaces: Rea, Proc. of CHI 99, Pittsburgh, PA, ACM, 1999.

Digital Library

[3]

Cassell, J., Stone, M., & Yan, H., Coordination and context-dependence in the generation of embodied conversation, Proc. INLG 2000, Mitzpe Ramon, Israel, 2000.

Digital Library

[4]

Cassell, J. and Thorisson, K. R., The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents, Applied Art. Intell., vol. 13, pp. 519--538, 1999.

[5]

Cassell, J., Torres, O., & Prevost, S., Turn Taking vs. Discourse Structure: How Best to Model Multimodal Conversation., in Machine Conversations, Y. Wilks, Ed. The Hague: Kluwer, 1999, pp. 143--154.

[6]

Cassell, J., Vilhjálmasson, H., & Bickmore, T., BEAT: The Behavior Expression Animation Toolkit, Proc. of SIGGRAPH, ACM Press, 2001.

Digital Library

[7]

Chovil, N., Discourse-Oriented Facial Displays in Conversation, Research on Language and Social Interaction, vol. 25, pp. 163--194, 1992.

[8]

Chu-Carroll, J. & Brown, M., Initiative in Collaborative Interactions - Its Cues and Effects, Proc. of AAAI Spring 1997 Symp. on Computational Models of Mixed Initiative, 1997.

[9]

Condon, W. S. & Osgton, W. D., Speech and body motion synchrony of the speaker-hearer, in The perception of language, D. Horton & J. Jenkins, Eds. NY: Academic Press, 1971, pp. 150--184.

[10]

Duncan, S., On the structure of speaker-auditor interaction during speaking turns, Language in Society, vol. 3, pp. 161--180, 1974.

[11]

Green, N., Carenini, G., Kerpedjiev, S., & Roth, S, A Media-Independent Content Language for Integrated Text and Graphics Generation, Proc. of Workshop on Content Visualization and Intermedia Representations at COLING and ACL '98, 1998.

[12]

Grosz, B. & Sidner, C., Attention, Intentions, and the Structure of Discourse, Computational Linguistics, vol. 12, pp. 175--204, 1986.

Digital Library

[13]

Kendon, A., Some Relationships between Body Motion and Speech, in Studies in Dyadic Communication, A. W. Siegman and B. Pope, Eds. Elmsford, NY: Pergamon Press, 1972, pp. 177--210.

[14]

Lesh, N., Rich, C., & Sidner, C., Using Plan Recognition in Human-Computer Collaboration, Proc. of the Conference on User Modelling, Banff, Canada, NY: Springer Wien, 1999.

Digital Library

[15]

Lester, J., Towns, S., Callaway, C., Voerman, J., & FitzGerald, P., Deictic and Emotive Communication in Animated Pedagogical Agents, in Embodied Conversational Agents, J. Cassell, J. Sullivan, et. al, Eds. Cambridge: MIT Press, 2000.

Digital Library

[16]

Lochbaum, K., A Collaborative Planning Model of Intentional Structure, Computational Linguistics, vol. 24, pp. 525--572, 1998.

Digital Library

[17]

McNeill, D., Hand and Mind: What Gestures Reveal about Thought. Chicago, IL/London, UK: The University of Chicago Press, 1992.

[18]

Rich, C. & Sidner, C. L., COLLAGEN: A Collaboration Manager for Software Interface Agents, User Modeling and User-Adapted Interaction, vol. 8, pp. 315--350, 1998.

Digital Library

[19]

Rickel, J. & Johnson, W. L., Task-Oriented Collaboration with Embodied Agents in Virtual Worlds, in Embodied Conversational Agents, J. Cassell, Ed. Cambridge, MA: MIT Press, 2000.

Digital Library

[20]

Sidner, C., An Artificial Discourse Language for Collaborative Negotiation, Proc. of 12th Intnl. Conf. on Artificial Intelligence (AAAI), Seattle, WA, MIT Press, 1994.

Digital Library

[21]

Takeuchi, A. & Nagao, K., Communicative facial displays as a new conversational modality, Proc. of InterCHI '93, Amsterdam, NL, ACM, 1993.

Digital Library

[22]

Thompson, L. and Massaro, D., Evaluation and Integration of Speech and Pointing Gestures during Referential Understanding, Journal of Experimental Child Psychology, vol. 42, pp. 144--168, 1986.

Cited By

Pan YAgrawal RSingh K(2024)S3: Speech, Script and Scene driven Head and Eye AnimationACM Transactions on Graphics10.1145/365817243:4(1-12)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658172
Ehret JBönsch ANossol PErmert CMohanathasan CSchlittmeier SFels JKuhlen TLugrin BLatoschik Mvon Mammen SKopp SPécune FPelachaud C(2023)Who's next?Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents10.1145/3570945.3607312(1-8)Online publication date: 19-Sep-2023
https://dl.acm.org/doi/10.1145/3570945.3607312
Cuadra ABaek HEstrin DJung MDell N(2022)On Inclusion: Video Analysis of Older Adult Interactions with a Multi-Modal Voice Assistant in a Public SettingProceedings of the 2022 International Conference on Information and Communication Technologies and Development10.1145/3572334.3572371(1-17)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3572334.3572371
Show More Cited By

Recommendations

Multimodal Collaboration in Expository Discourse: Verbal and Nonverbal Moves Alignment
Speech and Computer
Abstract
The paper explores multimodal collaboration in expository discourse considering verbal and nonverbal moves used by the participants in turn-taking. It reports the results of an experiment which tested speech, gesture and gaze alignment as affected ...
Graph representations of discourse structure

The aim of this paper is to introduce a new method to represent discourse structures. This approach is based on three discourse theories in order to highlight three discourse features: cohesion, coherence and intentionality. The graph representations of ...
Attention, intentions, and the structure of discourse

In this paper we explore a new theory of discourse structure that stresses the role of purpose and processing in discourse. In this theory, discourse structure is composed of three separate but interrelated components: the structure of the sequence of ...

Comments

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics

July 2001

562 pages

General Chair:
Bonnie Lynn Webber

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 06 July 2001

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
1,955
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)16

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pan YAgrawal RSingh K(2024)S3: Speech, Script and Scene driven Head and Eye AnimationACM Transactions on Graphics10.1145/365817243:4(1-12)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658172
Ehret JBönsch ANossol PErmert CMohanathasan CSchlittmeier SFels JKuhlen TLugrin BLatoschik Mvon Mammen SKopp SPécune FPelachaud C(2023)Who's next?Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents10.1145/3570945.3607312(1-8)Online publication date: 19-Sep-2023
https://dl.acm.org/doi/10.1145/3570945.3607312
Cuadra ABaek HEstrin DJung MDell N(2022)On Inclusion: Video Analysis of Older Adult Interactions with a Multi-Modal Voice Assistant in a Public SettingProceedings of the 2022 International Conference on Information and Communication Technologies and Development10.1145/3572334.3572371(1-17)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3572334.3572371
Robe PKuttal S(2022)Designing PairBuddy—A Conversational Agent for Pair ProgrammingACM Transactions on Computer-Human Interaction10.1145/349832629:4(1-44)Online publication date: 5-May-2022
https://dl.acm.org/doi/10.1145/3498326
Chen YYen C(2021)Understanding the influence of stress on sedentary workers’ sitting behavior in screen-based interaction contextAdjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447527.3474854(1-5)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447527.3474854
Perez-Navarro AGarcia VConesa J(2021)Students perception of videos in introductory physics courses of engineering in face-to-face and online environmentsMultimedia Tools and Applications10.1007/s11042-020-09665-080:1(1009-1028)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1007/s11042-020-09665-0
Bracewell DTomlinson MBrunson MPlymale JBracewell JBoerger aIde NXia F(2012)Annotation of adversarial and collegial social actions in discourseProceedings of the Sixth Linguistic Annotation Workshop10.5555/2392747.2392776(184-192)Online publication date: 12-Jul-2012
https://dl.acm.org/doi/10.5555/2392747.2392776
Mutlu BKanda TForlizzi JHodgins JIshiguro H(2012)Conversational gaze mechanisms for humanlike robotsACM Transactions on Interactive Intelligent Systems10.1145/2070719.20707251:2(1-33)Online publication date: 13-Jan-2012
https://dl.acm.org/doi/10.1145/2070719.2070725
Schulman DBickmore T(2011)Posture, relationship, and discourse structure models of nonverbal behavior for long-term interactionProceedings of the 10th international conference on Intelligent virtual agents10.5555/2041666.2041680(106-112)Online publication date: 15-Sep-2011
https://dl.acm.org/doi/10.5555/2041666.2041680
Endrass BAndré ERehm MLipi ANakano YSonenberg LStone PTumer KYolum P(2011)Culture-related differences in aspects of behavior for virtual characters across Germany and JapanThe 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 210.5555/2031678.2031680(441-448)Online publication date: 2-May-2011
https://dl.acm.org/doi/10.5555/2031678.2031680
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents