research-article

Simulating Human Imprecision in Temporal Statements of Intelligent Virtual Agents

Authors:

Susanne Schmidt,

Sven Zimmermann,

Frank SteinickeAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 422, Pages 1 - 15

https://doi.org/10.1145/3491102.3517625

Published: 29 April 2022 Publication History

Abstract

Research on intelligent virtual agents (IVAs) often concerns the implementation of human-like behavior by integrating artificial intelligence algorithms. Thus far, few studies focused on mimicry of cognitive imperfections inherent to humans in IVAs. Neglecting to implement such imperfect behavior in IVAs might result in less believable or engaging human-agent interactions. In this paper, we simulate human imprecision in conversational IVAs’ temporal statements. We conducted a survey to identify temporal statement patterns, transferred them to a conversational IVA, and conducted a user study evaluating the effects of time precision on perceived anthropomorphism and usefulness. Statistical analyses reveal significant interaction between time precision and agents’ use of memory aids, indicating that (i) imprecise agents are perceived as more human-like than precise agents when responding immediately, and (ii) unnaturally high levels of temporal precision can be compensated for by memory aid use. Further findings underscore the value of continued inquiry into cultural variations.

Supplemental Material

MP4 File

Video Preview

Transcript for: Video Preview

ZIP File

Supplemental Materials

Download
400.56 KB

References

[1]

David Ahn, Sisay Fissaha Adafre, and Maarten de Rijke. 2005. Extracting Temporal Information from Open Domain Text: A Comparative Exploration. Journal of Digital Information Management 3, 1 (2005), 14–20.

[2]

Jan Allbeck and Norman I Badler. 2001. Consistent Communication with Control. In Proceedings of the Workshop on Non-Verbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents at the Conference on Autonomous Agents (AGENTS). ACM, New York, NY, US, 1–6.

[3]

Andreas Armborst. 2017. Thematic Proximity in Content Analysis. Sage Open 7, 2 (2017), 1–11.

[4]

Christoph Bartneck, Dana Kulić, Elizabeth Croft, and Susana Zoghbi. 2009. Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots. International Journal of Social Robotics 1, 1 (2009), 71–81.

[5]

Timothy Bickmore, Daniel Schulman, and Langxuan Yin. 2009. Engagement vs. Deceit: Virtual Humans with Human Autobiographies. In Proceedings of the International Conference on Intelligent Virtual Agents (IVA). Springer, Berlin, Heidelberg, Germany, 6–19.

Digital Library

[6]

Richard A Block. 2003. Psychological Timing Without a Timer: The Roles of Attention and Memory. In Time and Mind II: Information Processing Perspectives. Hogrefe & Huber Publishers, Göttingen, Germany, 41–59.

[7]

Susan E Brennan and Maurice Williams. 1995. The Feeling of Another’s Knowing: Prosody and Filled Pauses as Cues to Listeners About the Metacognitive States of Speakers. Journal of Memory and Language 34, 3 (1995), 383–398.

[8]

Stephen Buetow and Katharine Wallis. 2019. The Beauty in Perfect Imperfection. Journal of Medical Humanities 40, 3 (2019), 389–394.

[9]

Ondřej Burkert, Cyril Brom, Rudolf Kadlec, and Jiří Lukavskỳ. 2010. Timing in Episodic Memory: Virtual Characters in Action. In Remembering Who We Are – Human Memory for Artificial Agents Symposium. De Montfort University, Leicester, UK, 1–8.

[10]

James Carifio and Rocco Perla. 2008. Resolving the 50-Year Debate Around Using and Misusing Likert Scales. Medical Education 42, 12 (2008), 1150–1152.

[11]

W Crawford. 2008. Place and Time Adverbials in Native and Non-Native English Student Writing. In Corpora and Discourse: The Challenges of Different Settings. John Benjamins Publishing Company, Amsterdam, Netherlands, 267–289.

[12]

David Crystal. 1966. Specification and English Tenses. Journal of Linguistics 2, 1 (1966), 1–34.

[13]

DeepL. 2021. DeepL Translate: The World’s Most Accurate Translator. https://www.deepl.com/en/translator. Accessed: 2021-12-29.

[14]

Virginie Demeure, Radosław Niewiadomski, and Catherine Pelachaud. 2011. HHow Is Believability of a Virtual Agent Related to Warmth, Competence, Personification, and Embodiment?Presence 20, 5 (2011), 431–448.

[15]

Joao Dias, Wan Ching Ho, Thurid Vogt, Nathalie Beeckman, Ana Paiva, and Elisabeth André. 2007. I Know What I Did Last Summer: Autobiographic Memory in Synthetic Characters. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII). Springer, Berlin, Heidelberg, Germany, 606–617.

Digital Library

[16]

Alexander Eiselmayer, Chat Wacharamanotham, Michel Beaudouin-Lafon, and Wendy E. Mackay. 2019. Touchstone2: An Interactive Environment for Exploring Trade-Offs in HCI Experiment Design. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). ACM, New York, NY, US, 1–11.

Digital Library

[17]

Eisko. 2018. Animatable Digital Double of Louise by Eisko©. www.eisko.com

[18]

Thomas Erickson, Catalina M. Danis, Wendy A. Kellogg, and Mary E. Helander. 2008. Assistance: The Work Practices of Human Administrative Assistants and Their Implications for IT and Organizations. In Proceedings of the Conference on Computer Supported Cooperative Work (CSCW). ACM, New York, NY, US, 609–618.

Digital Library

[19]

Lisa Ferro. 2001. TIDES Instruction Manual for the Annotation of Temporal Expressions. Technical Report. The MITRE Corporation.

[20]

W. Friedman. 2005. Developmental and Cognitive Perspectives on Humans’ Sense of the Times of Past and Future Events. Learning and Motivation 36 (2005), 145–158.

[21]

William J Friedman. 2004. Time in Autobiographical Memory. Social Cognition 22, 5: Special issue (2004), 591–605.

[22]

Wael H Gomaa and Aly A Fahmy. 2013. A Survey of Text Similarity Approaches. international Journal of Computer Applications 68, 13(2013), 13–18.

[23]

P Gomes, Ana Paiva, and Carlos Martinho. 2010. Episodic Memory Retrieval Through Re-appraisal of Past Events. In Proceedings of the International Conference on Intelligent Virtual Agents (IVA) - Supplementary Materials. Springer, Philadelphia, US, 26–27.

[24]

Paulo F Gomes, Carlos Martinho, and Ana Paiva. 2011. I’ve Been Here Before! Location and Appraisal in Memory Retrieval. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) - Volume 3. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, US, 1039–1046.

[25]

Google. 2021. Dialogflow | Google Cloud. cloud.google.com/dialogflow/

[26]

Google. 2021. Google Translate. https://translate.google.com/. Accessed: 2021-12-29.

[27]

Google. 2021. Speech-to-Text: Automatic Speech Recognition | Google Cloud. cloud.google.com/speech-to-text/

[28]

Google. 2021. Text-to-Speech: Lifelike Speech Synthesis | Google Cloud. cloud.google.com/text-to-speech/

[29]

Robert J Graham. 1981. The Role of Perception of Time in Consumer Research. Journal of Consumer Research 7, 4 (1981), 335–342.

[30]

David Graus, Paul N Bennett, Ryen W White, and Eric Horvitz. 2016. Analyzing and Predicting Task Reminders. In Proceedings of the Conference on User Modeling Adaptation and Personalization (UMAP). ACM, New York, NY, US, 7–15.

Digital Library

[31]

Simon Grondin. 2010. Timing and Time Perception: A Review of Recent Behavioral and Neuroscience Findings and Theoretical Directions. Attention, Perception, & Psychophysics 72, 3 (2010), 561–582.

[32]

Charles Hampden-Turner, Fons Trompenaars, and Charles Hampden-Turner. 2020. Riding the Waves of Culture: Understanding Diversity in Global Business. Nicholas Brealey Publishing, Boston, MA, US.

[33]

Martin Haspelmath. 1997. From Space to Time. Lincom Europa, München, Germany, Newcastle, UK.

[34]

Wan Ching Ho and Kerstin Dautenhahn. 2008. Towards a Narrative Mind: The Creation of Coherent Life Stories for Believable Virtual Agents. In Proceedings of the International Conference on Intelligent Virtual Agents (IVA). Springer, Berlin, Heidelberg, Germany, 59–72.

Digital Library

[35]

Margaret J Intons-Peterson and JoAnne Fournier. 1986. External and Internal Memory Aids: When and How Often Do We Use Them?Journal of Experimental Psychology: General 115, 3 (1986), 267.

[36]

Margaret Jean Intons-Peterson and George L Newsome. 1992. External Memory Aids: Effects and Effectiveness. In Memory Improvement. Springer, New York, NY, US, 101–121.

[37]

Steve MJ Janssen, Antonio G Chessa, and Jaap MJ Murre. 2006. Memory for Time: How People Date Events. Memory & Cognition 34, 1 (2006), 138–147.

[38]

Ryota Kanai, Mia Yuan Dong, Bahador Bahrami, and Geraint Rees. 2011. Distractibility in Daily Life Is Reflected in the Structure and Function of Human Parietal Cortex. Journal of Neuroscience 31, 18 (2011), 6620–6626.

[39]

Zerrin Kasap, Maher Ben Moussa, Parag Chaudhuri, and Nadia Magnenat-Thalmann. 2009. Making Them Remember—Emotional Virtual Characters With Memory. Computer Graphics and Applications 29, 2 (2009), 20–29.

Digital Library

[40]

Charlyn M Laserna, Yi-Tai Seih, and James W Pennebaker. 2014. Um... Who Like Says You Know: Filler Word Use as a Function of Age, Gender, and Personality. Journal of Language and Social Psychology 33, 3 (2014), 328–338.

[41]

Douglas B Lenat. 2009. Building a Machine Smart Enough to Pass the Turing Test. In Parsing the Turing Test. Springer, Dordrecht, Netherlands, 261–282.

[42]

Willem JM Levelt, Graham Richardson, and Wido La Heij. 1985. Pointing and Voicing in Deictic Expressions. Journal of Memory and Language 24, 2 (1985), 133–164.

[43]

Robert V Levine and Ara Norenzayan. 1999. The Pace of Life in 31 Countries. Journal of Cross-Cultural Psychology 30, 2 (1999), 178–205.

[44]

Microsoft. 2021. Windows Virtual Desktop | Remote Desktop | Microsoft Azure. azure.microsoft.com/en-us/services/virtual-desktop/

[45]

George A Miller. 1995. WordNet: A Lexical Database for English. Commun. ACM 38, 11 (1995), 39–41.

Digital Library

[46]

Miro. 2021. The Visual Collaboration Platform for Every Team: Miro. https://miro.com/. Accessed: 2021-12-29.

[47]

Geoff Norman. 2010. Likert Scales, Levels of Measurement and the “Laws” of Statistics. Advances in Health Sciences Education 15, 5 (2010), 625–632.

[48]

Nahal Norouzi, Gerd Bruder, Brandon Belna, Stefanie Mutter, Damla Turgut, and Greg Welch. 2019. A Systematic Review of the Convergence of Augmented Reality, Intelligent Virtual Agents, and the Internet of Things. In Artificial Intelligence in IoT. Springer, Cham, 1–24.

[49]

Nahal Norouzi, Kangsoo Kim, Jason Hochreiter, Myungho Lee, Salam Daher, Gerd Bruder, and Greg Welch. 2018. A Systematic Survey of 15 Years of User Studies Published in the Intelligent Virtual Agents Conference. In Proceedings of the International Conference on Intelligent Virtual Agents (IVA). ACM, New York, NY, US, 17–22.

Digital Library

[50]

Rafael E Núñez, Benjamin A Motz, and Ursina Teuscher. 2006. Time After Time: The Psychological Reality of the Ego-and Time-Reference-Point Distinction in Metaphorical Construals of Time. Metaphor and Symbol 21, 3 (2006), 133–146.

[51]

Asher R Pacht. 1984. Reflections on Perfection. American Psychologist 39, 4 (1984), 386.

[52]

Deborah Richards and Karla Bransky. 2014. Forgetmenot: What and How Users Expect Intelligent Virtual Agents to Recall and Forget Personal Conversational Content. International Journal of Human-Computer Studies 72, 5 (2014), 460–476.

[53]

José I Rojas-Méndez, Gary Davies, Omer Omer, Paitoon Chetthamrongchai, and Canan Madran. 2002. A Time Attitude Scale for Cross Cultural Research. Journal of Global Marketing 15, 3-4 (2002), 117–147.

[54]

Xin Rong, Adam Fourney, Robin N Brewer, Meredith Ringel Morris, and Paul N Bennett. 2017. Managing Uncertainty in Time Expressions for Virtual Assistants. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). ACM, New York, NY, US, 568–579.

Digital Library

[55]

Roser Saurı, Jessica Littman, Bob Knippen, Robert Gaizauskas, Andrea Setzer, and James Pustejovsky. 2006. TimeML Annotation Guidelines Version 1.2.1.

[56]

Susanne Schmidt, Oscar Ariza, and Frank Steinicke. 2020. Intelligent Blended Agents: Reality–Virtuality Interaction with Artificially Intelligent Embodied Virtual Humans. Multimodal Technologies and Interaction 4, 4 (2020), 85.

[57]

Susanne Schmidt, Gerd Bruder, and Frank Steinicke. 2018. Effects of Embodiment on Generic and Content-Specific Intelligent Virtual Agents as Exhibition Guides. In Proceedings of the International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE). Eurographics Association, Geneve, Switzerland, 13–20.

[58]

Susanne Schmidt, Oscar Javier Ariza Nunez, and Frank Steinicke. 2019. Blended Agents: Manipulation of Physical Objects within Mixed Reality Environments and Beyond. In Proceedings of the Symposium on Spatial User Interaction (SUI). ACM, New York, NY, US, 1–10.

Digital Library

[59]

Anna Sircova, Fons JR Van De Vijver, Evgeny Osin, Taciano L Milfont, Nicolas Fieulaine, Altinay Kislali-Erginbilgic, Philip G Zimbardo, 2015. Time Perspective Profiles of Cultures. In Time Perspective Theory; Review, Research and Application. Springer, Cham, Switzerland, 169–187.

[60]

Carlota S Smith. 1978. The Syntax and Interpretation of Temporal Expressions in English. Linguistics and Philosophy 2, 1 (1978), 43–99.

[61]

SurveyCircle. 2021. The Largest Community for Online Research. www.surveycircle.com/

[62]

Charles P Thompson. 1982. Memory for Unique Personal Events: The Roommate Study. Memory & Cognition 10, 4 (1982), 324–332.

[63]

Charles P Thompson, John J Skowronski, and D John Lee. 1988. Telescoping in Dating Naturally Occurring Events. Memory & Cognition 16, 5 (1988), 461–468.

[64]

Catherine Tinsley. 1998. Models of Conflict Resolution in Japanese, German, and American Cultures. Journal of Applied Psychology 83, 2 (1998), 316.

[65]

Hegler Tissot, Marcos Didonet Del Fabro, Leon Derczynski, and Angus Roberts. 2019. Normalisation of Imprecise Temporal Expressions Extracted From Text. Knowledge and Information Systems 61, 3 (2019), 1361–1394.

Digital Library

[66]

Jean-Claude G Usunier. 1991. Business Time Perceptions and National Cultures: A Comparative Survey. MIR: Management International Review 31, 3 (1991), 197–217.

[67]

Bart Van Den Hooff. 2004. Electronic Coordination and Collective Action: Use and Effects of Electronic Calendaring and Scheduling. Information & Management 42, 1 (2004), 103–114.

Digital Library

[68]

Isaac Wang, Jesse Smith, and Jaime Ruiz. 2019. Exploring Virtual Agents for Augmented Reality. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). ACM, New York, NY, US, 1–12.

Digital Library

[69]

Dan Zakay and Richard A Block. 2004. Prospective and Retrospective Duration Judgments: An Executive-Control Perspective. Acta Neurobiologiae Experimentalis 64, 3 (2004), 319–328.

[70]

Gal Zauberman, B Kyu Kim, Selin A Malkoc, and James R Bettman. 2009. Discounting Time and Time Discounting: Subjective Time Perception and Intertemporal Preferences. Journal of Marketing Research 46, 4 (2009), 543–556.

[71]

Qing Zhou and Richard Fikes. 2002. A Reusable Time Ontology. In Proceeding of the AAAI Workshop on Ontologies for the Semantic Web. The AAAI Press, Menlo Park, CA, US, 35–40.

[72]

XiaoJia Zhou, HaoMin Li, XuDong Lu, and HuiLong Duan. 2011. Temporal Expression Recognition and Temporal Relationship Extraction from Chinese Narrative Medical Records. In Proceedings of the International Conference on Bioinformatics and Biomedical Engineering (ICBBE). IEEE, New York, NY, US, 1–4.

Cited By

Schmidt SRolff TVoigt HOffe MSteinicke F(2024)Natural Expression of a Machine Learning Model's Uncertainty Through Verbal and Non-Verbal Behavior of Intelligent Virtual AgentsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676454(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676454
Pillis DPataranutaporn PMaes PSra M(2024)AI Comes Out of the Closet: Using AI-Generated Virtual Characters to Help Individuals Practice LGBTQIA+ AdvocacyProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645213(686-698)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645213
Dunnell KAgarwal GPataranutaporn PLippman AMaes P(2024)AI-Generated Media for Exploring Alternate RealitiesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650861(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650861
Show More Cited By

Index Terms

Simulating Human Imprecision in Temporal Statements of Intelligent Virtual Agents
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Governing intelligent virtual agent behaviour with norms
AAMAS '13: Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems

One requirement by which virtual environments (VEs) are judged, is the believability of the virtual agents (VAs). One aspect of believability, is that agent responses to situations should not create cognitive dissonance and thereby distract the ...
Expectation and complex event handling in BDI-based intelligent virtual agents (demonstration)
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3

When operating in virtual communities, intelligent agents should maintain a high-level awareness of the physical and social environment around them in order to be more believable and capable. However, due to the inherent differences between virtual ...
Agent communication for believable human-like interactions between virtual characters
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3

In this paper we present a model for realizing believable human-like interaction between virtual agents situated cognitively in a MAS on one side while embodied in a virtual environment within a game engine on the other side. A middleware approach is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

German Federal Ministry of Education and Research (BMBF)

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
401
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)4

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Schmidt SRolff TVoigt HOffe MSteinicke F(2024)Natural Expression of a Machine Learning Model's Uncertainty Through Verbal and Non-Verbal Behavior of Intelligent Virtual AgentsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676454(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676454
Pillis DPataranutaporn PMaes PSra M(2024)AI Comes Out of the Closet: Using AI-Generated Virtual Characters to Help Individuals Practice LGBTQIA+ AdvocacyProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645213(686-698)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645213
Dunnell KAgarwal GPataranutaporn PLippman AMaes P(2024)AI-Generated Media for Exploring Alternate RealitiesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650861(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650861
Cao QYu HCharisse PQiao SStevens B(2023)Is High-Fidelity Important for Human-like Virtual Avatars in Human Computer Interactions?International Journal of Network Dynamics and Intelligence10.53941/ijndi0201008(15-23)Online publication date: 27-Mar-2023
https://doi.org/10.53941/ijndi0201008

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten