Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Conversational gaze mechanisms for humanlike robots

Published: 13 January 2012 Publication History

Abstract

During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.

References

[1]
Argyle, M. and Dean, J. 1965. Eye-contact, distance and affiliation. Sociometry 28, 3, 289--304.
[2]
Argyle, M. and Ingham, R. 1972. Gaze, mutual gaze, and proximity. Semiotica 6, 1, 32--49.
[3]
Aron, A., Aron, E., and Smollan, D. 1992. Inclusion of other in the self scale and the structure of interpersonal closeness. J. Person. Soc. Psych. 63, 4, 596--612.
[4]
Bailenson, J., Beall, A., Loomis, J., Blascovich, J., and Turk, M. 2005. Transformed social interaction, augmented gaze, and social influence in immersive virtual environments. Hum. Comm. Resear. 31, 4, 511--537.
[5]
Bales, R. 1970. Personality and Interpersonal Behavior. Holt, Rinehart, and Winston, New York.
[6]
Bales, R., Strodtbeck, F., Mills, T., and Roseborough, M. 1951. Channels of communication in small groups. Amer. Soc. Rev. 16, 4, 461--468.
[7]
Bennewitz, M., Faber, F., Joho, D., Schreiber, M., and Behnke, S. 2006. Towards a humanoid museum guide robot that interacts with multiple persons. In Proceedings of the 5th IEEE-RAS International Conference on Humanoid Robots. IEEE, 418--423.
[8]
Bouman, C. 1997. Cluster: An unsupervised algorithm for modeling Gaussian mixtures. http://www. ece.purdue.edu/~bouman.
[9]
Brown, G., Currie, K., and Kenworthy, J. 1980. Questions of Intonation. Routledge.
[10]
Brown, P. and Levinson, S. 1987. Politeness: Some Universals in Language Usage. Cambridge University Press.
[11]
Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., and Yan, H. 1999a. Embodiment in conversational interfaces: Rea. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 520--527.
[12]
Cassell, J., Nakano, Y., Bickmore, T., Sidner, C., and Rich, C. 2001. Non-verbal cues for discourse structure. In Proceedings of the 39th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 114--123.
[13]
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., and Stone, M. 1994. Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques. ACM, 413--420.
[14]
Cassell, J., Torres, O., and Prevost, S. 1999b. Turn taking vs. discourse structure: How best to model multimodal conversation. In Machine Conversations, Kluwer, 143--154.
[15]
Clark, H. 1992. Arenas of Language Use. University of Chicago Press.
[16]
Clark, H. 1996. Using Language. Cambridge University Press.
[17]
Clark, H. and Carlson, T. 1982. Hearers and speech acts. Language 58, 2, 332--373.
[18]
Colburn, A., Cohen, M., and Drucker, S. 2000. The role of eye gaze in avatar mediated conversational interfaces. Tech. rep. MSR-TR-2000-81, Microsoft Research.
[19]
Cook, M. and Smith, J. 1975. The role of gaze in impression formation. Brit. J. Soc. Clin. Psych. 14, 19--25.
[20]
Duncan, S. 1972. Some signals and rules for taking speaking turns in conversations. J. Person. Soc. Psych. 23, 2, 283--292.
[21]
Edelsky, C. 1981. Who's got the floor? Lang. Soc. 10, 03, 383--421.
[22]
Efran, J. 1968. Looking for approval: effects on visual behavior of approbation from persons differing in importance. J. Person. Soc. Psych. 10, 1, 21--25.
[23]
Exline, R. 1963. Explorations in the process of person perception: visual interaction in relation to competition, sex, and need for affiliation1. J. Person. 31, 1, 1--20.
[24]
Garau, M., Slater, M., Bee, S., and Sasse, M. 2001. The impact of eye gaze on communication using humanoid avatars. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 309--316.
[25]
Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., and Sasse, M. 2003. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 529--536.
[26]
Goffman, E. 1955. On face-work; an analysis of ritual elements in social interaction. Psych. Interper. Biol. Proc. 18, 3, 213--231.
[27]
Goffman, E. 1971. Relations in Public: Microstudies of the Public Order. Harper & Row.
[28]
Goffman, E. 1979. Footing. Semiotica 25, 1-2, 1--30.
[29]
Goldberg, L., Johnson, J., Eber, H., Hogan, R., Ashton, M., Cloninger, C., and Gough, H. 2006. The international personality item pool and the future of public-domain personality measures. J. Resear. Person. 40, 1, 84--96.
[30]
Goodwin, C. 1980. Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning. Soc. Inq. 50, 3-4, 272--302.
[31]
Goodwin, C. 1981. Conversational Organization: Interaction between Speakers and Hearers. Academic Press New York.
[32]
Grosz, B. and Sidner, C. 1986. Attention, intentions, and the structure of discourse. Computat. Linguist. 12, 3, 175--204.
[33]
Halliday, M. 1967. Intonation and Grammar in British English. Mouton.
[34]
Hanks, W. 1996. Language & Communicative Practices. Westview Press.
[35]
Hanna, J. and Brennan, S. 2007. Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. J. Mem. Lang. 57, 4, 596--615.
[36]
Hayashi, R. 1988. Simultaneous talkÑfrom the perspective of floor management of English and Japanese speakers. World Englishes 7, 3, 269--288.
[37]
Heylen, D., Es, I., Nijholt, A., and Dijk, B. 2005. Controlling the gaze of conversational agents. In Advances in Natural Multimodal Dialogue Systems, N. Ide, J. Véronis, H. Baayen, K. Church, J. Klavans, D. Barnard, D. Tufis, J. Llisterri, S. Johansson, J. Mariani, J. Kuppevelt, L. Dybkjær, and N. Bernsen, Eds. Text, Speech and Language Technology Series, vol. 30, Springer, Berlin, 245--262.
[38]
Hinds, J. 1976. Aspects of Japanese Discourse Structure. Kaitakusha.
[39]
Hirschberg, J. and Grosz, B. 1992. Intonational features of local and global discourse structure. In Proceedings of the Workshop on Speech and Natural Language. Association for Computational Linguistics, 441--446.
[40]
Hirschberg, J. and Pierrehumbert, J. 1986. The intonational structuring of discourse. In Proceedings of the 24th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 136--144.
[41]
Hymes, D. 1972. Models of the Interaction of Language and Social Life. Holt, Rinehalt & Winston, 35--77.
[42]
Ishiguro, H., Ono, T., Imai, M., Maeda, T., Kanda, T., and Nakatsu, R. 2001. Robovie: An interactive humanoid robot. Indust. Robot. A Int. J. 28, 6, 498--504.
[43]
Kendon, A. 1967. Some functions of gaze-direction in social interaction. Acta Psychologica 26, 1, 22.
[44]
Kirchner, N., Alempijevic, A., and Dissanayake, G. 2011. Nonverbal robot-group interaction using an imitated gaze cue. In Proceedings of the 6th International Conference on Human-Robot Interaction. ACM, 497--504.
[45]
Kleck, R. and Nuessle, W. 1968. Congruence between the indicative and communicative functions of eye contact in interpersonal relations. Brit. J. Soc. Clin. Psych. 7, 241--246.
[46]
Kuno, Y., Sadazuka, K., Kawashima, M., Yamazaki, K., Yamazaki, A., and Kuzuoka, H. 2007. Museum guide robot based on sociological interaction analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1191--1194.
[47]
Laurel, B. 1991. Computers as Theatre. Addison-Wesley.
[48]
Lee, S., Badler, J., and Badler, N. 2002. Eyes alive. ACM Trans. Graph. 21. ACM, 637--644.
[49]
Levinson, S. 1988. Putting Linguistics on a Proper Footing: Explorations in Goffman's Concepts of Participation. 161--227, Oxford, UK, Polity Press.
[50]
Libby, W. 1970. Eye contact and direction of looking as stable individual differences. J. Exper. Resear. Person. 4, 303--312.
[51]
Lu, D., Pileggi, A., Wilson, C., and Smart, W. 2010. What Can Actors Teach Robots About Interaction? In Proceedings of AAAI Spring Symposium Series.
[52]
Mason, M., Tatkow, E., and Macrae, C. 2005. The look of love. Psych. Sci. 16, 3, 236--239.
[53]
Maynard, S. 1986. On back-channel behavior in Japanese and English casual conversation. Linguistics 24, 6, 1079--1108.
[54]
Maynard, S. 1989. Japanese Conversation: Self-Contextualization through Structure and Interactional Management. Ablex Publishing, Norwood, NJ.
[55]
McLaughlin, M. and Cody, M. 1982. Awkward silences: Behavioral antecedents and consequences of the conversational lapse. Hum. Comm. Resear. 8, 4, 299--316.
[56]
Murray, N., Roberts, D., Steed, A., Sharkey, P., Dickerson, P., and Rae, J. 2007. An assessment of eye-gaze potential within immersive virtual environments. ACM Trans. Multimed. Comput. Comm. Appl. 3, 4, 8.
[57]
Mutlu, B., Forlizzi, J., and Hodgins, J. 2006. A storytelling robot: Modeling and evaluation of human-like gaze behavior. In Proceedings of the 6th IEEE-RAS International Conference on Humanoid Robots. IEEE, 518--523.
[58]
Nielsen, G. 1962. Studies in Self Confrontation. Munksgaard, Copenhagen.
[59]
Otteson, J. and Otteson, C. 1980. Effect of teacher's gaze on children's story recall. Percept. Motor Skills 50, 35--42.
[60]
Parise, S., Kiesler, S., Sproull, L., and Waters, K. 1996. My partner is a real dog: cooperation with social agents. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. ACM, 399--408.
[61]
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X., Kirbas, C., McCullough, K., and Ansari, R. 2002a. Multimodal human discourse: gesture and speech. ACM Trans. Comput.-Hum. Interac. 9, 3, 171--193.
[62]
Quek, F., McNeill, D., Bryll, R., Kirbas, C., Arslan, H., McCullough, K., Furuyama, N., and Ansari, R. 2000. Gesture, speech, and gaze cues for discourse segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognitionn. Vol. 2, 247--254.
[63]
Quek, F., McNeill, D., Bryll, R., Kirbas, C., Arslan, H., McCullough, K., Furuyama, N., and Ansari, R. 2002b. Gesture, speech, and gaze cues for discourse segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vol. 2, IEEE, 247--254.
[64]
Rehm, M. and André, E. 2005. Where do they look? Gaze behaviors of multiple users interacting with an embodied conversational agent. In Intelligent Virtual Agents, Springer, 241--252.
[65]
Sacks, H., Schegloff, E., and Jefferson, G. 1974. A simplest systematics for the organization of turn-taking for conversation. Language 50, 4, 696--735.
[66]
Schegloff, E. 1968. Sequencing in Conversational Openings. Amer. Anthro. 70, 6, 1075--1095.
[67]
Schegloff, E. 2000. Overlapping talk and the organization of turn-taking for conversation. Lang. Soc. 29, 1, 1--63.
[68]
Schegloff, E. and Sacks, H. 1973. Opening up closings. Semiotica 8, 4, 289--327.
[69]
Schiffrin, D. 1988. Discourse Markers. Cambridge University Press.
[70]
Sherwood, J. 1987. Facilitative effects of gaze upon learning. Percept. Motor Skills 64, 1275--1278.
[71]
Sidner, C., Kidd, C., Lee, C., and Lesh, N. 2004. Where to look: a study of human-robot engagement. In Proceedings of the 9th International Conference on Intelligent User Interfaces. ACM, 78--84.
[72]
Staudte, M. and Crocker, M. 2011. Investigating joint attention mechanisms through spoken human-robot interaction. Cognition 120, 268--291.
[73]
Steptoe, W., Wolff, R., Murgia, A., Guimaraes, E., Rae, J., Sharkey, P., Roberts, D., and Steed, A. 2008. Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. ACM, 197--200.
[74]
Tanaka, H. 1999. Turn-Taking in Japanese Conversation: A Study in Grammar and Interaction. Vol. 3, John Benjamins Publishing Company.
[75]
Tannen, D. 2005. Conversational Style: Analyzing Talk Among Friends. Oxford University Press.
[76]
Thomas, F. and Johnston, O. 1995. The Illusion of Life: Disney Animation. Hyperion New York.
[77]
Thórisson, K. 2002. Natural turn-taking needs no manual: Computational theory and model, from perception to action. In Multimodality in Language and Speech Systems, Kluwer, 173--207.
[78]
Trafton, J., Bugajska, M., Fransen, B., and Ratwani, R. 2008. Integrating vision and audition within a cognitive architecture to track conversations. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction. ACM, 201--208.
[79]
Van Breemen, A. 2004. Bringing robots to life: Applying principles of animation to robots. In Proceedings of Shaping Human-Robot Interaction Workshop at the 22nd ACM/SigCHI Conference on Human Factors in Computing.
[80]
Vertegaal, R., Slagter, R., van der Veer, G., and Nijholt, A. 2001. Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 301--308.
[81]
Vertegaal, R., van der Veer, G., and Vons, H. 2000. Effects of gaze on multiparty mediated communication. In Proceedings of Graphics Interface. 95--102.
[82]
Vilhjálmsson, H. and Cassell, J. 1998. Bodychat: Autonomous communicative behaviors in avatars. In Proceedings of the 2nd International Conference on Autonomous Agents. ACM, 269--276.
[83]
Wang, N. and Johnson, W. 2008. The Politeness Effect in an intelligent foreign language tutoring system. In Intelligent Tutoring Systems, Springer, 270--280.
[84]
Wang, N., Johnson, W., Rizzo, P., Shaw, E., and Mayer, R. 2005. Experimental evaluation of polite interaction tactics for pedagogical agents. In Proceedings of the 10th International Conference on Intelligent User Interfaces. ACM, 12--19.
[85]
Ward, N. and Tsukahara, W. 2000. Prosodic features which cue back-channel responses in English and Japanese* 1. J. Pragmatics 32, 8, 1177--1207.
[86]
Watson, D., Clark, L., and Tellegen, A. 1988. Development and validation of brief measures of positive and negative affect: The PANAS scales. J. Person. Soc. Psych. 54, 6, 1063--1070.
[87]
Watson, O. 1970. Proxemic behavior: A cross-cultural study. Mouton, The Hague.
[88]
Weisbrod, R. 1965. Looking behavior in a discussion group. Unpublished manuscript. Cornell University.
[89]
Whittaker, S. and Stenton, P. 1988. Cues and control in expert-client dialogues. In Proceedings of the 26th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 123--130.
[90]
Wilkes-Gibbs, D. and Clark, H. 1992. Coordinating beliefs in conversation. J. Mem. Lang. 31, 2, 183--194.
[91]
Williams, K., Cheung, C., and Choi, W. 2000. Cyberostracism: Effects of being ignored over the Internet. J. Person. Soc. Psych. 79, 5, 748--762.
[92]
Wirth, J., Sacco, D., Hugenberg, K., and Williams, K. 2010. Eye gaze as relational evaluation: Averted eye gaze leads to feelings of ostracism and relational devaluation. Personal. Soc. Psych. Bull. 36, 7, 869--882.
[93]
Yamazaki, A., Yamazaki, K., Burdelski, M., Kuno, Y., and Fukushima, M. 2010. Coordination of verbal and non-verbal actions in human-robot interaction at museums and exhibitions. J. Pragmatics 42, 9, 2398--2414.
[94]
Yamazaki, A., Yamazaki, K., Kuno, Y., Burdelski, M., Kawashima, M., and Kuzuoka, H. 2008. Precision timing in human-robot interaction: coordination of head movement and utterance. In Proceeding of the 26th SIGCHI Conference on Human Factors in Computing Systems. ACM, 131--140.
[95]
Yngve, V. 1970. On getting a word in edgewise. In Proceedings of the 6th Regional Meeting of the Chicago Linguistic Society. 657--677.

Cited By

View all
  • (2024)Analysis of Gaze Behaviors in Multi-Party Dialogues and Personality Expression by Gaze Motions Generated in Interactive Robots複数人対話における視線動作の解析および対話ロボットの視線動作生成による個性の表出Journal of the Robotics Society of Japan10.7210/jrsj.42.15142:2(151-158)Online publication date: 2024
  • (2024)Performing a Task Alongside a Robot: Exploring the Impact of Social ComparisonExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651055(1-7)Online publication date: 11-May-2024
  • (2024)Probing the Inductive Biases of a Gaze Model for Multi-party InteractionCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640702(507-511)Online publication date: 11-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Interactive Intelligent Systems
ACM Transactions on Interactive Intelligent Systems  Volume 1, Issue 2
January 2012
157 pages
ISSN:2160-6455
EISSN:2160-6463
DOI:10.1145/2070719
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2012
Accepted: 01 October 2011
Revised: 01 August 2011
Received: 01 December 2010
Published in TIIS Volume 1, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Human-robot interaction
  2. conversation
  3. conversational roles
  4. discourse structure
  5. gaze
  6. humanlike robots
  7. turn-taking

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)122
  • Downloads (Last 6 weeks)14
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Analysis of Gaze Behaviors in Multi-Party Dialogues and Personality Expression by Gaze Motions Generated in Interactive Robots複数人対話における視線動作の解析および対話ロボットの視線動作生成による個性の表出Journal of the Robotics Society of Japan10.7210/jrsj.42.15142:2(151-158)Online publication date: 2024
  • (2024)Performing a Task Alongside a Robot: Exploring the Impact of Social ComparisonExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651055(1-7)Online publication date: 11-May-2024
  • (2024)Probing the Inductive Biases of a Gaze Model for Multi-party InteractionCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640702(507-511)Online publication date: 11-Mar-2024
  • (2024)A Social Approach for Autonomous Vehicles: A Robotic Object to Enhance Passengers' Sense of Safety and TrustProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634998(86-95)Online publication date: 11-Mar-2024
  • (2024)Understanding Large-Language Model (LLM)-powered Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634966(371-380)Online publication date: 11-Mar-2024
  • (2024)Scarecrows in Oz: The Use of Large Language Models in HRIACM Transactions on Human-Robot Interaction10.1145/360626113:1(1-11)Online publication date: 31-Mar-2024
  • (2024)Can You Fill Me? a Coffee Machine With Robotic Behavior That Promotes Prosocial Behavior2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)10.1109/RO-MAN60168.2024.10731323(2044-2050)Online publication date: 26-Aug-2024
  • (2024) Where and When Should the Teleoperated Avatar Look: Gaze Instruction Dataset for Enhanced Teleoperated Avatar Communication * 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS58592.2024.10802049(2494-2501)Online publication date: 14-Oct-2024
  • (2024)Enhancing social robot's direct gaze expression through vestibulo-ocular movementsAdvanced Robotics10.1080/01691864.2024.239855638:19-20(1457-1469)Online publication date: 10-Sep-2024
  • (2024)Gaze modeling in multi-party dialogues and extraversion expression through gaze aversion controlAdvanced Robotics10.1080/01691864.2024.239453838:19-20(1470-1485)Online publication date: 10-Sep-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media