research-article

Open access

Human, Chameleon or Nodding Dog?

Authors:

Patrick G.T. HealeyAuthors Info & Claims

ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

Pages 428 - 436

https://doi.org/10.1145/3242969.3242998

Published: 02 October 2018 Publication History

Abstract

Immersive virtual environments (IVEs) present rich possibilities for the experimental study of non-verbal communication. Here, the 'digital chameleon' effect, -which suggests that a virtual speaker (agent) is more persuasive if they mimic their addresses head movements-, was tested. Using a specially constructed IVE, we recreate a full-body analogue version of the 'digital chameleon' experiment. The agent's behaviour is manipulated in three conditions 1) Mimic (Chameleon) in which it copies the participant's nodding 2) Playback (Nodding Dog) which uses nods from playback of a previous participant and are therefore unconnected with the content and 3) Original (Human) in which it uses the prerecorded actor's movements. The results do not support the original finding of differences in ratings of agent persuasiveness between conditions. However, motion capture data reveals systematic differences in a) the real-time movements of speakers and listeners b) between the Original, Mimic and Playback conditions. We conclude that the automatic mimicry model is too simplistic and that this paradigm must address the reciprocal dynamics of non-verbal interaction to achieve its full potential.

References

[1]

Sun Joo Ahn, Jeremy N. Bailenson, and Dooyeon Park . 2014. Short- and long-term effects of embodied experiences in immersive virtual environments on environmental locus of control and behavior. Computers in Human Behavior Vol. 39, February (2014), 235--245.

Digital Library

[2]

Sun Joo (Grace) Ahn, Amanda Minh Tran Le, and Jeremy Bailenson . 2013. The Effect of Embodied Experiences on Self-Other Merging, Attitude, and Helping Behavior. Media Psychology 785022367 (2013).

[3]

Jens Allwood and Loredana Cerrato . 2003. A study of gestural feedback expressions. In First nordic symposium on multimodal communication. Copenhagen, 7--22.

[4]

Ferran Argelaguet . 2016. The Role of Interaction in Virtual Embodiment : Effects of the Virtual Hand Representation The Role of Interaction in Virtual Embodiment : Effects of the Virtual Hand Representation., bibinfonumpages3 -- 10 pages.

[5]

Jeremy N Bailenson, Jim Blascovich, Andrew C Beall, and Jack M Loomis . 2001 a. Equilibrium theory revisited: Mutual gaze and personal space in virtual environments. Presence: Teleoperators & Virtual Environments Vol. 10, 6 (2001), 583--598.

Digital Library

[6]

Jeremy N. Bailenson, Jim Blascovich, Andrew C. Beall, and Jack M. Loomis . 2001 b. Equilibrium Theory Revisited: Mutual Gaze and Personal Space in Virtual Environments. Presence: Teleoperators and Virtual Environments Vol. 10, 6 (2001), 583--598. {arxiv}arXiv:1011.1669v3

Digital Library

[7]

Jeremy N. Bailenson, Jim Blascovich, Andrew C. Beall, and Jack M. Loomis . 2003. Interpersonal Distances in Virtual Environments. Personality and Social Psychology Bulletin Vol. 29, 7 (2003), 819--833.

[8]

Jeremy N. Bailenson and Nick Yee . 2003. The Use of Immersive Virtual Reality in the Learning Sciences: Digital Transformations of Teachers, Students, and Social Context. Journal of the Learning Sciences Vol. 12, 3 (2003), 361--407.

[9]

Jeremy N Bailenson and Nick Yee . 2005. Digital Chameleons. Society Vol. 16, 10 (2005), 814--819.

[10]

Jeremy N. Bailenson and Nick Yee . 2007. Virtual interpersonal touch and digital chameleons. Journal of Nonverbal Behavior Vol. 31, 4 (2007), 225--242.

[11]

Stuart A Battersby and Patrick G T Healey . 2010 a. Head and Hand Movements in the Orchestration of Dialogue. Proceedings of the Cognitive Science Society Vol. 32, 32 (jan . 2010). deftempurl%http://escholarship.org/uc/item/4v80m010 tempurl

[12]

Stuart A Battersby and Patrick G T Healey . 2010 b. Using head movement to detect listener responses during multi-party dialogue Proceedings of LREC workshop on multimodal corpora advances in capturing coding and analysing multimodality. 11--15.

[13]

Janet Beavin Bavelas, Linda Coates, and Trudy Johnson . 2000. Listeners as co-narrators. Journal of personality and social psychology Vol. 79, 6 (2000), 941.

[14]

Janet Beavin Bavelas and Sara Healing . 2013. Reconciling the effects of mutual visibility on gesturing: A review. Gesture Vol. 13, 1 (2013), 63--92.

[15]

Ilias Bergström, Konstantina Kilteni, and Mel Slater . 2016. First-person perspective virtual body posture influences stress: a virtual reality body ownership study. PloS one Vol. 11, 2 (2016), e0148060.

[16]

Jim Blascovich, Jack Loomis, Andrew C Beall, Kimberly R Swinth, Crystal L Hoyt, and Jeremy N Bailenson . 2002. Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology. Vol. 13, 2 (2002), 103--124.

[17]

Max Boholm and Jens Allwood . 2010. Repeated head movements, their function and relation to speech Proceedings of LREC workshop on multimodal corpora advances in capturing coding and analysing multimodality. Citeseer, 6--10.

[18]

Judee K Burgoon, Thomas Birk, and Michael Pfau . 1990. Nonverbal behaviors, persuasion, and credibility. Human communication research Vol. 17, 1 (1990), 140--169.

[19]

Caroline Catmur and Cecilia Heyes . 2013. Is it what you do, or when you do it? The roles of contingency and similarity in pro-social effects of imitation. January (2013), 1--25.

[20]

Tanya L. Chartrand and John A. Bargh . 1999. The chameleon effect: The perception--behavior link and social interaction. Journal of Personality and Social Psychology Vol. 76, 6 (1999), 893--910.

[21]

Tanya L Chartrand and Jessica L Lakin . 2013. The antecedents and consequences of human behavioral mimicry. Annual review of psychology Vol. 64 (2013), 285--308.

[22]

Rick Dale and Michael J Spivey . 2006. Unraveling the dyad: Using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Language Learning Vol. 56, 3 (2006), 391--430.

[23]

Joyce A Edinger and Miles L Patterson . 1983. Nonverbal involvement and social control. Psychological Bulletin Vol. 93, 1 (1983), 30.

[24]

Charles Goodwin . 1979. The Interactive Construction of a Sentence in Natural Conversation. (1979), 97--121. deftempurl%http://www.sscnet.ucla.edu/clic/cgoodwin/79inter_sent.pdf tempurl

[25]

Allison Gray . 2016. Cross Recurrence Quantification Analysis Tutorial. deftempurl%https://quantdev.ssri.psu.edu/tutorials/cross-recurrence-quantification-analysis Retrieved 22 April, 2018 from tempurl

[26]

Rosanna E Guadagno, Jim Blascovich, Jeremy N Bailenson, and Cade McCall . 2007. Virtual humans and persuasion: The effects of agency and behavioral realism. Media Psychology Vol. 10, 1 (2007), 1--22.

[27]

Uri Hadar, Timothy J Steiner, and F Clifford Rose . 1985. Head movement during listening turns in conversation. Journal of Nonverbal Behavior Vol. 9, 4 (1985), 214--228.

[28]

Joanna Hale and Antonia F. De C. Hamilton . 2016. Testing the relationship between mimicry, trust and rapport in virtual reality conversations. Scientific Reports Vol. 6 (2016).

[29]

Patrick G. T. Healey, Christopher Frauenberger, Marco Gillies, and Stuart Battersby . 2009. Experimenting with Non-Verbal Interaction. (2009), 2--3.

[30]

Patrick G T Healey, Mary Lavelle, Christine Howes, Stuart Battersby, and Rosemarie McCabe . 2013. How listeners respond to speaker's troubles. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. Vol. 35.

[31]

Patrick G T Healey, Nicola Plant, Christine Howes, and Mary Lavelle . 2015. When words fail: collaborative gestures during clarification dialogues 2015 AAAI Spring Symposium Series,(Chicago).

[32]

Dirk Heylen . 2005. Challenges ahead: Head movements and other social acts in conversations Proceedings of Joint Symposium on Virtual Social Agents. 45--52.

[33]

Dirk Heylen . 2006. Head gestures, gaze and the principles of conversational structure. International Journal of Humanoid Robotics Vol. 3, 03 (2006), 241--267.

[34]

Carlos Toshinori Ishi, Hiroshi Ishiguro, and Norihiro Hagita . 2014. Analysis of relationship between head motion events and speech in dialogue conversations. Speech Communication Vol. 57 (2014), 233--243.

Digital Library

[35]

Adam Kendon . 1990. Conducting interaction: Patterns of behavior in focused encounters. Vol. Vol. 7. CUP Archive.

[36]

Anna K Kuhlen and Susan E Brennan . 2013 a. Language in dialogue: when confederates might be hazardous to your data. Psychonomic bulletin & review Vol. 20, 1 (2013), 54--72.

[37]

Anna K. Kuhlen and Susan E. Brennan . 2013 b. Language in dialogue: when confederates might be hazardous to your data. Psychonomic Bulletin & Review Vol. 20, 1 (2013), 54--72.

[38]

Marc Erich Latoschik, Daniel Roth, Dominik Gall, Jascha Achenbach, Thomas Waltemate, and Mario Botsch . 2017. The effect of avatar realism in immersive social virtual realities Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. ACM, 39.

Digital Library

[39]

Jina Lee, Zhiyang Wang, and Stacy Marsella . 2010. Evaluating models of speaker head nods for virtual agents Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 1257--1264.

Digital Library

[40]

Antonella Maselli and Mel Slater . 2013. The building blocks of the full body ownership illusion. Frontiers in human neuroscience Vol. 7 (2013), 83.

[41]

Marianne Schmid Mast and Gaëtan Cousin . 2013. Power, Dominance, and Persuasion. In Nonverbal Communication (Handbooks of Communication Science, HOCS 2), bibfieldeditorJudith A Hall and Mark L Knapp (Eds.). DE GRUYTER, Berlin, Boston, Chapter 20, 613--635.

[42]

Albert Mehrabian . 2017. Nonverbal communication. Routledge.

[43]

Albert Mehrabian and Martin Williams . 1969. Nonverbal concomitants of perceived and intended persuasiveness. Journal of Personality and Social psychology Vol. 13, 1 (1969), 37.

[44]

Daniel C Richardson and Rick Dale . 2005. Looking to understand: The coupling between speakers' and listeners' eye movements and its relationship to discourse comprehension. Cognitive science Vol. 29, 6 (2005), 1045--1060.

[45]

Michael J Richardson, Stacy Lopresti-Goodman, Marisa Mancini, Bruce Kay, and RC Schmidt . 2008. Comparing the attractor strength of intra-and interpersonal interlimb coordination using cross-recurrence analysis. Neuroscience Letters Vol. 438, 3 (2008), 340--345.

[46]

Laurel D Riek, Philip C Paul, and Peter Robinson . 2010. When my robot smiles at me: Enabling human-robot rapport via real-time head gesture mimicry. Journal on Multimodal User Interfaces Vol. 3, 1--2 (2010), 99--108.

[47]

Albert E Scheflen . 1964. The significance of posture in communication systems. Psychiatry Vol. 27, 4 (1964), 316--331.

[48]

Catherine J Stevens, Bronwyn Pinchbeck, Trent Lewis, Martin Luerssen, Darius Pfitzner, David MW Powers, Arman Abrahamyan, Yvonne Leung, and Guillaume Gibert . 2016. Mimicry and expressiveness of an ECA in human-agent interaction: familiarity breeds content! Computational cognitive science Vol. 2, 1 (2016), 1.

[49]

Frank M F Verberne, Jaap Ham, Aditya Ponnada, and Cees J H Midden . 2013. Trusting Digital Chameleons : The Effect of Mimicry. (2013), 234--245.

Digital Library

[50]

Charles L Webber Jr and Joseph P Zbilut . 2005. Recurrence quantification analysis of nonlinear dynamical systems. Tutorials in contemporary nonlinear methods for the behavioral sciences (2005), 26--94.

[51]

Marcin Włodarczak, Hendrik Buschmeier, Zofia Malisz, Stefan Kopp, and Petra Wagner . 2012. Listener head gestures and verbal feedback expressions in a distraction task Feedback Behaviors in Dialog.

Cited By

Ratcliffe JTokarchuk L(2022)The potential of remote XR experimentation: Defining benefits and limitations through expert survey and case studyFrontiers in Computer Science10.3389/fcomp.2022.9529964Online publication date: 2-Dec-2022
https://doi.org/10.3389/fcomp.2022.952996
Mills GGregoromichelaki EHowes CMaraev V(2022)Influencing laughter with AI-mediated communicationInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.00011.mil22:3(416-463)Online publication date: 28-Mar-2022
https://doi.org/10.1075/is.00011.mil
Ratcliffe JSoave FBryan-Kinns NTokarchuk LFarkhatdinov IKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Extended Reality (XR) Remote Research: a Survey of Drawbacks and OpportunitiesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445170(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445170
Show More Cited By

Index Terms

Human, Chameleon or Nodding Dog?

Recommendations

Analysis of Human Nodding Behavior during Group Work for Designing Nodding Robots
GROUP '16: Proceedings of the 2016 ACM International Conference on Supporting Group Work

Nodding has various communicative functions in humans, such as agreement, emphasis and turn-taking and can also create various positive impressions in communication by the person exhibiting the behavior. The ultimate aim of our research is to facilitate ...
An event-based architecture to manage virtual human non-verbal communication in 3d chatting environment
AMDO'12: Proceedings of the 7th international conference on Articulated Motion and Deformable Objects

Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world ...
Speaker Motion Patterns during Self-repairs in Natural Dialogue
ICMI '22 Companion: Companion Publication of the 2022 International Conference on Multimodal Interaction

An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

October 2018

687 pages

ISBN:9781450356923

DOI:10.1145/3242969

General Chairs:
Sidney K. D'Mello
University of Illinois, USA
,
Panayiotis (Panos) Georgiou
University of Southern California, USA
,
Stefan Scherer
University of Southern California, USA
,
Program Chairs:
Emily Mower Provost
University of Michigan, USA
,
Mohammad Soleymani
University of Southern California, USA
,
Marcelo Worsley
Northwestern University, USA

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: Specialist Interest Group in Computer-Human Interaction of the ACM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

EPSRC and AHRC Centre for Doctoral Training in Media and Arts Technology

Conference

ICMI '18

Sponsor:

SIGCHI

ICMI '18: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 16 - 20, 2018

CO, Boulder, USA

Acceptance Rates

ICMI '18 Paper Acceptance Rate 63 of 149 submissions, 42%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
703
Total Downloads

Downloads (Last 12 months)171
Downloads (Last 6 weeks)35

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ratcliffe JTokarchuk L(2022)The potential of remote XR experimentation: Defining benefits and limitations through expert survey and case studyFrontiers in Computer Science10.3389/fcomp.2022.9529964Online publication date: 2-Dec-2022
https://doi.org/10.3389/fcomp.2022.952996
Mills GGregoromichelaki EHowes CMaraev V(2022)Influencing laughter with AI-mediated communicationInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.00011.mil22:3(416-463)Online publication date: 28-Mar-2022
https://doi.org/10.1075/is.00011.mil
Ratcliffe JSoave FBryan-Kinns NTokarchuk LFarkhatdinov IKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Extended Reality (XR) Remote Research: a Survey of Drawbacks and OpportunitiesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445170(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445170
Ratcliffe JSoave FHoover MOrtega FBryan-Kinns NTokarchuk LFarkhatdinov I(2021)Remote XR Studies: Exploring Three Key Challenges of Remote XR ExperimentationExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3442472(1-4)Online publication date: 8-May-2021
https://dl.acm.org/doi/10.1145/3411763.3442472
Jonell PElkind EVeloso MAgmon NTaylor M(2019)Using Social and Physiological Signals for User Adaptation in Conversational AgentsProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3306127.3332133(2420-2422)Online publication date: 8-May-2019
https://dl.acm.org/doi/10.5555/3306127.3332133

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents