research-article

“I See You!”: A Design Framework for Interface Cues about Agent Visual Perception from a Thematic Analysis of Videogames

Authors:

Matthew Rueben,

Matthew R Horrocks,

Jennifer Eleanor Martinez,

Michelle V Cormier,

Nicolas LaLone,

Marlena Fraune, and

Phoebe O. Toups DugasAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

Article No.: 417, Pages 1 - 22

https://doi.org/10.1145/3491102.3517699

Published: 29 April 2022 Publication History

Abstract

As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can “see.” Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types – both visualizations and sonifications – available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

Supplementary Material

MP4 File (3491102.3517699-talk-video.mp4)

Talk Video

Download
202.70 MB

References

[1]

Henny Admoni, Bradley Hayes, David Feil-Seifer, Daniel Ullman, and Brian Scassellati. 2013. Are you looking at me? Perception of robot attention is mediated by gaze type and group size. In 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 389–395.

[2]

Sultan A. Alharthi, Olaa Alsaedi, Phoebe O. Toups Dugas, Theresa Jean Tanenbaum, and Jessica Hammer. 2018. Playing to Wait: A Taxonomy of Idle Games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). ACM, New York, NY, USA, 621:1–621:15.

Digital Library

[3]

Sultan A. Alharthi, Phoebe O. Toups Dugas, Olaa Alsaedi, Theresa Jean Tanenbaum, and Jessica Hammer. 2018. The Pleasure of Playing Less: A Study of Incremental Games through the Lens of Kittens. Well Played Singles(2018). http://dx.doi.org/https://doi.org/10.1184/R1/6686957.v1

[4]

Mahoro Anabuki, Hiroyuki Kakuta, Hiroyuki Yamamoto, and Hideyuki Tamura. 2000. Welbo: an embodied conversational agent living in mixed reality space. In Extended Abstracts on Human Factors in Computing Systems (CHI ’00). ACM Press, The Hague, The Netherlands, 10. http://dx.doi.org/10.1145/633292.633299

Digital Library

[5]

Ronald T. Azuma. 1997. A Survey of Augmented Reality. In Presence: Teleoperators and Virtual Environments, Vol. 6. 355–385.

[6]

Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2–11. Issue: 1.

[7]

Christoph Bartneck, Tony Belpaeme, Friederike Eyssel, Takayuki Kanda, Merel Keijsers, and Selma Šabanović. 2020. Human-robot interaction: An introduction. Cambridge University Press.

[8]

Peter Bayliss. 2007. Beings in the Game-World: Characters, Avatars, and Players. In Proceedings of the 4th Australasian Conference on Interactive Entertainment(IE ’07). RMIT University, Melbourne, AUS.

[9]

Victoria Bellotti, Maribeth Back, W. Keith Edwards, Rebecca E. Grinter, Austin Henderson, and Cristina Lopes. 2002. Making Sense of Sensing Systems: Five Questions for Designers and Researchers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’02). Association for Computing Machinery, New York, NY, USA, 415–422. http://dx.doi.org/10.1145/503376.503450

Digital Library

[10]

Steve Benford, Andy Crabtree, Martin Flintham, Adam Drozd, Rob Anastasi, Mark Paxton, Nick Tandavanitj, Matt Adams, and Ju Row-Farr. 2006. Can You See Me Now?ACM Trans. Comput.-Hum. Interact. 13, 1 (March 2006), 100–133. http://dx.doi.org/10.1145/1143518.1143522

Digital Library

[11]

Steve Benford and Gabriella Giannachi. 2011. Performing Mixed Reality. MIT Press.

Digital Library

[12]

Staffan Björk and Jussi Holopainen. 2006. Games and Design Patterns. In The Game Design Reader: A Rules of Play Anthology, Katie Salen and Eric Zimmerman (Eds.). 410–437.

[13]

Gabriele Bolano, Christian Juelg, Arne Roennau, and Ruediger Dillmann. 2019. Transparent Robot Behavior Using Augmented Reality in Close Human-Robot Interaction. In 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 1–7. http://dx.doi.org/10.1109/RO-MAN46459.2019.8956296 ISSN: 1944-9437.

Digital Library

[14]

Nathan Bos, Ann Zimmerman, Judith Olson, Jude Yew, Jason Yerkie, Erik Dahl, and Gary Olson. 2007. From Shared Databases to Communities of Practice: A Taxonomy of Collaboratories. Journal of Computer-Mediated Communication 12, 2 (2007), 652–672.

[15]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. http://dx.doi.org/10.1191/1478088706qp063oa

[16]

Anke M. Brock, Julia Chatain, Michelle Park, Tommy Fang, Martin Hachet, James A. Landay, and Jessica R. Cauchard. 2018. FlyMap: Interacting with Maps Projected from a Drone. In Proceedings of the 7th ACM International Symposium on Pervasive Displays(PerDis ’18). Association for Computing Machinery, New York, NY, USA. http://dx.doi.org/10.1145/3205873.3205877

Digital Library

[17]

Janis A Cannon-Bowers, Eduardo Salas, and Sharolyn Converse. 1993. Shared Mental Models in Expert Team Decision Making. In Individual and Group Decision Making: Current Issues, N John Castellan Jr (Ed.). Earlbaum Associates, Hillsdale, NJ, USA, 221–246.

[18]

John M Carroll and Mary Beth Rosson. 1987. Paradox of the active user. In Interfacing thought: Cognitive aspects of human-computer interaction. 80–111.

[19]

Marcus Carter, Martin Gibbs, and Michael Arnold. 2012. Avatars, Characters, Players and Users: Multiple Identities at/in Play. In Proceedings of the 24th Australian Computer-Human Interaction Conference(OzCHI ’12). Association for Computing Machinery, New York, NY, USA, 68–71. http://dx.doi.org/10.1145/2414536.2414547

Digital Library

[20]

Justine Cassell. 2000. Nudge nudge wink wink: Elements of face-to-face conversation for embodied conversational agents. In Embodied conversational agents, Justine Cassell, Joseph Sullivan, Elizabeth Churchill, and Scott Prevost (Eds.). Vol. 1. MIT Press.

[21]

Justine Cassell and Kristinn R. Thorisson. 1999. The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence 13, 4-5 (May 1999), 519–538. http://dx.doi.org/10.1080/088395199117360

[22]

Elizabeth Cha, Anca D. Dragan, and Siddhartha S. Srinivasa. 2015. Perceived robot capability. In 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 541–548.

Digital Library

[23]

Elizabeth Cha, Yunkyung Kim, Terrence Fong, and Maja J. Mataric. 2018. A survey of nonverbal signaling methods for non-humanoid robots. Foundations and Trends® in Robotics 6, 4 (2018), 211–323. Publisher: Now Publishers, Inc.

[24]

Matthew Chalmers and Areti Galani. 2004. Seamful interweaving: Heterogeneity in the theory and design of interactive systems. In DIS ’04: Proceedings of the 2004 Conference on Designing Interactive Systems. ACM Press, 243–252. http://dx.doi.org/10.1145/1013115.1013149

Digital Library

[25]

Jessie Y.C. Chen and Michael J. Barnes. 2015. Agent Transparency for Human-Agent Teaming Effectiveness. In IEEE International Conference on Systems, Man, and Cybernetics. 1381–1385. http://dx.doi.org/10.1109/SMC.2015.245

Digital Library

[26]

Hao-Fei Cheng, Ruotong Wang, Zheng Zhang, Fiona O’Connell, Terrance Gray, F. Maxwell Harper, and Haiyi Zhu. 2019. Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow, Scotland, UK, 1–12. http://dx.doi.org/10.1145/3290605.3300789

Digital Library

[27]

Victor Cheung, Y.-L. Betty Chang, and Stacey D. Scott. 2012. Communication channels and awareness cues in collocated collaborative time-critical gaming. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. 569–578.

Digital Library

[28]

Martin Constable, Justin Dauwels, Shoko Dauwels, Rasheed Umer, Mengyu Zhou, and Yasir Tahir. 2016. Modelling Conversation. In Context Aware Human-Robot and Human-Agent Interaction, Nadia Magnenat-Thalmann, Junsong Yuan, Daniel Thalmann, and Bum-Jae You (Eds.). Springer International Publishing, Cham, 81–111. http://dx.doi.org/10.1007/978-3-319-19947-4_4

[29]

Rajkumar Darbar, Joan Sol Roo, Thibault Lainé, and Martin Hachet. 2019. DroneSAR: Extending Physical Spaces in Spatial Augmented Reality Using Projection on a Drone. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia(MUM ’19). Association for Computing Machinery, New York, NY, USA. http://dx.doi.org/10.1145/3365610.3365631

Digital Library

[30]

Drew Davidson. 2008. Well Played: Interpreting Prince of Persia: The Sands of Time. Games and Culture 3, 3-4 (July 2008), 356–386.

[31]

Leslie A DeChurch and Jessica R. Mesmer-Magnus. 2010. Measuring shared team mental models: A meta-analysis. Group Dynamics: Theory, Research, and Practice 14, 1(2010), 1–14.

[32]

Anca D Dragan, Kenton CT Lee, and Siddhartha S Srinivasa. 2013. Legibility and predictability of robot motion. In 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 301–308.

Digital Library

[33]

Katharina Emmerich and Maic Masuch. 2016. The Influence of Virtual Agents on Player Experience and Performance. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’16). Association for Computing Machinery, New York, NY, USA, 10–21. http://dx.doi.org/10.1145/2967934.2968092

Digital Library

[34]

Katharina Emmerich, Patrizia Ring, and Maic Masuch. 2018. I’m Glad You Are on My Side: How to Design Compelling Game Companions. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’18). Association for Computing Machinery, New York, NY, USA, 141–152. http://dx.doi.org/10.1145/3242671.3242709

Digital Library

[35]

Mica R. Endsley. 1995. Toward a theory of situation awareness in dynamic systems. Human Factors 37, 1 (1995), 32–64.

[36]

Mica R. Endsley. 2000. Theoretical Underpinnings of Situation Awareness: A Critical Review. In Situation Awareness Analysis and Measurement, Mica R. Endsley and D J Garland (Eds.). Lawrence Erlbaum Associates, Mahwah, NJ, USA, 3–6.

[37]

Elliot E. Entin and Daniel Serfaty. 1999. Adaptive Team Coordination. Human Factors 41, 2 (June 1999), 312–325.

[38]

Michael R. Epke, Lars Kooijman, and Joost C. F. de Winter. 2021. I See Your Gesture: A VR-Based Study of Bidirectional Communication between Pedestrians and Automated Vehicles. Journal of Advanced Transportation 2021 (April 2021), e5573560. http://dx.doi.org/10.1155/2021/5573560 Publisher: Hindawi.

[39]

J. Alberto Espinosa, F. Javier Lerch, and Robert E. Kraut. 2004. Explicit versus implicit coordination mechanisms and task dependencies: One size does not fit all. In Team Cognition: Understanding the factors that Drive Process and Performance. American Psychological Association, 107–130.

[40]

Evelyn Florentine, Mark Adam Ang, Scott Drew Pendleton, Hans Andersen, and Marcelo H Ang Jr. 2016. Pedestrian notification methods in autonomous vehicles for multi-class mobility-on-demand service. In Proceedings of the Fourth International Conference on Human Agent Interaction. 387–392.

Digital Library

[41]

David Forsyth and Jean Ponce. 2011. Computer vision: A modern approach.Prentice hall.

[42]

D Gentner and A L Stevens. 1983. Mental Models. Lawrence Earlbaum Associates, Hillsdale, NJ, USA.

[43]

Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, Maria Ruiz, Sarah Miller, David R. Millen, Murray Campbell, Sadhana Kumaravel, and Wei Zhang. 2020. Mental Models of AI Agents in a Cooperative Game Setting. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–12. http://dx.doi.org/10.1145/3313831.3376316

Digital Library

[44]

Michael J. Gielniak and Andrea L. Thomaz. 2012. Enhancing Interaction through Exaggerated Motion Synthesis. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction(HRI ’12). Association for Computing Machinery, New York, NY, USA, 375–382. http://dx.doi.org/10.1145/2157689.2157813

Digital Library

[45]

Michael A Goodrich and Alan C Schultz. 2008. Human-robot interaction: a survey. Now Publishers Inc.

[46]

Andrea Gurtner, Franziska Tschan, Norbert K Semmer, and Christof Nägele. 2007. Getting groups to develop good strategies: Effects of reflexivity interventions on team process, team performance, and shared mental models. Organizational Behavior and Human Decision Process 102 (2007), 127–142.

[47]

Carl Gutwin and Saul Greenberg. 2004. The Importance of Awareness for Team Cognition in Distributed Collaboration. In Team Cognition: Understanding the Factors that Drive Process and Performance (1st ed.), Eduardo Salas and Stephen M. Fiore (Eds.). American Psychological Association, Washington, DC, USA, 177–201.

[48]

Komei Hasegawa and Yasushi Nakauchi. Preliminary Evaluation of a Telepresence Robot Conveying Pre-motions for Avoiding Speech Collisions. (????), 4.

[49]

Christian Heath and Paul Luff. 2000. Team work: Collaboration and control in London Underground line control rooms. In Technology in Action, Roy Pea, John Seely Brown, and Christian Heath (Eds.). Cambridge University Press, Cambridge, UK, 88–124.

[50]

Eva Hornecker, Paul Marshall, Nick Sheep Dalton, and Yvonne Rogers. 2008. Collaboration and Interference: Awareness with Mice or Touch Input. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work(CSCW ’08). Association for Computing Machinery, New York, NY, USA, 167–176. http://dx.doi.org/10.1145/1460563.1460589

Digital Library

[51]

Edwin Hutchins. 1988. The Technology of Team Navigation. ICS Report. Technology and Cooperative Work.

[52]

Ioanna Iacovides, Anna Cox, Richard Kennedy, Paul Cairns, and Charlene Jennett. 2015. Removing the HUD: The Impact of Non-Diegetic Game Elements and Expertise on Player Involvement. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’15). Association for Computing Machinery, New York, NY, USA, 13–22. http://dx.doi.org/10.1145/2793107.2793120

Digital Library

[53]

Colby Johanson and Regan L. Mandryk. 2016. Scaffolding Player Location Awareness through Audio Cues in First-Person Shooters. Association for Computing Machinery, New York, NY, USA, 3450–3461. https://doi.org/10.1145/2858036.2858172

Digital Library

[54]

Kristine Jørgensen. 2013. Gameworld Interfaces. MIT Press, Cambridge, MA, USA.

[55]

M. Waleed Kadous, Raymond Ka-Man Sheh, and Claude Sammut. 2006. Effective user interface design for rescue robotics. In Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. 250–257.

Digital Library

[56]

Sara Kiesler. 2005. Fostering common ground in human-robot interaction. In IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN). IEEE, 729–734.

[57]

Richard Klimoski and Susan Mohammed. 1994. Team Mental Model: Construct or Metaphor?Journal of Management 20, 2 (1994), 403–437.

[58]

Ross A Knepper, Christoforos I Mavrogiannis, Julia Proft, and Claire Liang. 2017. Implicit communication in a joint action. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (HRI). 283–292.

Digital Library

[59]

Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Austin Texas USA, 1–10. http://dx.doi.org/10.1145/2207676.2207678

Digital Library

[60]

Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, and Weng-Keen Wong. 2013. Too much, too little, or just right? Ways explanations impact end users’ mental models. In 2013 IEEE Symposium on Visual Languages and Human Centric Computing. IEEE, 3–10.

[61]

Johannes Kunkel, Tim Donkers, Lisa Michael, Catalin-Mihai Barbu, and Jürgen Ziegler. 2019. Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow, Scotland, UK, 1–12. http://dx.doi.org/10.1145/3290605.3300717

Digital Library

[62]

Nicolas LaLone. 2019. A Tale of Dungeons & Dragons and the Origins of the Game Platform. Analog Game Studies 3, 6 (2019).

[63]

Nicolas LaLone, Sultan A. Alharthi, and Phoebe O. Toups Dugas. 2019. A Vision of Augmented Reality for Urban Search and Rescue. In Proceedings of the Halfway to the Future Symposium 2019. 1–4.

[64]

Alex Leavitt, Brian C. Keegan, and Joshua Clark. 2016. Ping to Win? Non-Verbal Communication and Team Performance in Competitive Online Multiplayer Games. Association for Computing Machinery, New York, NY, USA, 4337–4350. https://doi.org/10.1145/2858036.2858132

Digital Library

[65]

Min Kyung Lee, Karen P. Tang, Jodi Forlizzi, and Sara Kiesler. 2011. Understanding users’ perception of privacy in human-robot interaction. In Proceedings of the 6th International Conference on Human-Robot Interaction (HRI). ACM, 181–182.

Digital Library

[66]

Sau-lai Lee, Ivy Yee-man Lau, Sara Kiesler, and Chi-Yue Chiu. 2005. Human mental models of humanoid robots. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2767–2772.

[67]

Michael Lewis, Huao Li, and Katia Sycara. 2021. Chapter 14 - Deep learning, transparency, and trust in human robot teamwork. In Trust in Human-Robot Interaction, Chang S. Nam and Joseph B. Lyons (Eds.). Academic Press, 321–352. http://dx.doi.org/10.1016/B978-0-12-819472-0.00014-9

[68]

Lucy R. Lippard. 1971. Marcel Duchamp. In Dadas on Art: Tzara, Arp, Duchamp, and Others, Lucy R. Lippard (Ed.). Dover Publications, Inc., 139–154.

[69]

Stefan Liszio, Katharina Emmerich, and Maic Masuch. 2017. The Influence of Social Entities in Virtual Reality Games on Player Experience and Immersion. In Proceedings of the 12th International Conference on the Foundations of Digital Games(FDG ’17). Association for Computing Machinery, New York, NY, USA. http://dx.doi.org/10.1145/3102071.3102086

Digital Library

[70]

Michal Luria. 2018. Designing Robot Personality Based on Fictional Sidekick Characters. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction(HRI ’18). Association for Computing Machinery, New York, NY, USA, 307–308. http://dx.doi.org/10.1145/3173386.3176912

Digital Library

[71]

Jean MacMillan, Elliot E. Entin, and Daniel Serfaty. 2004. Communication Overhead: The Hidden Cost of Team Cognition. In Team Cognition: Understanding the Factors that Drive Process and Performance. American Psychological Association, 61–82.

[72]

John E. Mathieu, Gerald F. Goodwin, Tonia S. Heffner, Eduardo Salas, and Janis A. Cannon-Bowers. 2000. The Influence of Shared Mental Models on Team Process and Performance. Journal of Applied Psychology 85, 2 (2000), 273–283.

[73]

Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE Trans. Information Systems E77-D, 12 (1994), 1321–1329.

[74]

Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (Feb. 2019), 1–38. http://dx.doi.org/10.1016/j.artint.2018.07.007

[75]

Brian Ka-Jun Mok, Stephen Yang, David Sirkin, and Wendy Ju. 2015. A place for every tool and every tool in its place: Performing collaborative tasks with interactive robotic drawers. In 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 700–706.

[76]

John Edison Muñoz Cardona and Kerstin Dautenhahn. 2021. Robo Ludens: Game Design Techniques Applied in HRI Experiments. In Companion of the ACM/IEEE International Conference on Human-Robot Interaction. 691–693.

Digital Library

[77]

Peter Naur. 1985. Programming as theory building. Microprocessing and microprogramming 15, 5 (1985), 253–261.

[78]

Don Norman. 2013. The Design of Everyday Things: Revised and Expanded Edition (2013 ed.). Basic Books, New York City, USA.

[79]

Oxford English Dictionary. 2017. “diegesis, n.”. Dictionary Definition. (June 2017). Retrieved September 15, 2017 from http://www.oed.com/view/Entry/52402.

[80]

Steffi Paepcke and Leila Takayama. 2010. Judging a bot by its cover: an experiment on expectation setting for personal robots. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE Press, 45–52.

[81]

Ramviyas Parasuraman, Sergio Caccamo, Fredrik Baaberg, Petter Ogren, and Mark Neerincx. 2017. A New UGV Teleoperation Interface for Improved Awareness of Network Connectivity and Physical Surroundings. Journal of Human-Robot Interaction 6, 3 (2017), 48–70.

Digital Library

[82]

Tomislav Pejsa, Sean Andrist, Michael Gleicher, and Bilge Mutlu. 2015. Gaze and Attention Management for Embodied Conversational Agents. ACM Transactions on Interactive Intelligent Systems 5, 1 (March 2015), 1–34. http://dx.doi.org/10.1145/2724731

Digital Library

[83]

Catherine Pelachaud. 2005. Multimodal expressive embodied conversational agents. In Proceedings of the 13th annual ACM international conference on Multimedia - MULTIMEDIA ’05. ACM Press, Hilton, Singapore, 683. http://dx.doi.org/10.1145/1101149.1101301

Digital Library

[84]

Muhammet Ramoğlu, Idil Bostan, Mohammad Obaid, Tilbe Göksun, and Oğuzhan Özcan. 2016. Interaction between Abstract Agents: Increasing the Readability of Causal Events with Animation Principles. In Proceedings of the 20th International Academic Mindtrek Conference(AcademicMindtrek ’16). Association for Computing Machinery, New York, NY, USA, 272–280. http://dx.doi.org/10.1145/2994310.2994345

Digital Library

[85]

Daniel J. Rea. 2020. Now You’re Teleoperating with Power: Learning from video games to improve teleoperation interfaces. Ph.D. Dissertation. University of Manitoba.

[86]

Tiago Ribeiro and Ana Paiva. 2012. The Illusion of Robotic Life: Principles and Practices of Animation for Robots. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction(HRI ’12). Association for Computing Machinery, New York, NY, USA, 383–390. http://dx.doi.org/10.1145/2157689.2157814

Digital Library

[87]

Justin Richer and Jill L. Drury. 2006. A video game-based framework for analyzing human-robot interaction: characterizing interface design in real-time interactive multimedia applications. In Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction. 266–273.

Digital Library

[88]

Anna L. Rowe and Nancy J. Cooke. 1995. Measuring Mental Models: Choosing the Right Tools for the Job.Human Resource Development Quarterly 6, 3 (1995), 243–255. http://lib-ezproxy.tamu.edu:2048/login?search.ebscohost.com/login.aspx?direct=true&db=bth&AN=14858776&site=ehost-live

[89]

Adam W. Ruch. 2011. Videogame interface: Artefacts and tropes. In Videogames Studies: Concepts, Cultures, and Communication. Brill, 1–9.

[90]

Matthew Rueben, Jeffrey Klow, Madelyn Duer, Eric Zimmerman, Jennifer Piacentini, Madison Browning, Frank J Bernieri, Cindy M Grimm, and William D Smart. 2021. Mental Models of a Mobile Shoe Rack: Exploratory Findings from a Long-term In-the-Wild Study. ACM Transactions on Human-Robot Interaction (THRI) 10, 2 (2021), 1–36.

Digital Library

[91]

Katie Salen and Eric Zimmerman. 2004. Rules of Play: Game Design Fundamentals. MIT Press, Cambridge, MA, USA.

Digital Library

[92]

Burkhard Schafer and Lilian Edwards. 2017. “I spy, with my little sensor”: fair data handling practices for robots between privacy, copyright and security. Connection Science 29, 3 (2017), 200–209.

Digital Library

[93]

Mark Scheeff, John Pinto, Kris Rahardja, Scott Snibbe, and Robert Tow. 2002. Experiences with Sparky, a social robot. In Socially Intelligent Agents. Springer, 173–180.

[94]

Matthias Scheutz, Scott A. DeLoach, and Julie A. Adams. 2017. A framework for developing and using shared mental models in human-agent teams. Journal of Cognitive Engineering and Decision Making 11, 3 (2017), 203–224.

[95]

Stela H. Seo, James E. Young, and Pourang Irani. 2017. Where are the robots? In-feed embedded techniques for visualizing robot team member locations. In 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 522–527.

Digital Library

[96]

Bruno Siciliano, Oussama Khatib, and Torsten Kröger. 2008. Springer handbook of robotics. Vol. 200. Springer.

[97]

Jan Smeddinck, Kamila Wajda, Adeel Naveed, Leen Touma, Yuting Chen, Muhammad Abu Hasan, Muhammad Waqas Latif, and Robert Porzel. 2010. QuickWoZ: a multi-purpose wizard-of-oz framework for experiments with embodied conversational agents. In Proceedings of the 15th international conference on Intelligent user interfaces - IUI ’10. ACM Press, Hong Kong, China, 427. http://dx.doi.org/10.1145/1719970.1720055

Digital Library

[98]

Lizzie Stark. 2012. We Hold these Rules to be Self-Evident: Larp as Metaphor for American Identity. States of Play: Nordic Larp around the World (2012), 184–189.

[99]

Renee J. Stout, Janis A. Cannon-Bowers, Eduardo Salas, and Dana M. Milanovich. 1999. Planning, shared mental models, and coordinated performance: An empirical link is established. Human Factors 41, 1 (March 1999), 61–71.

[100]

Leila Takayama, Doug Dooley, and Wendy Ju. 2011. Expressing Thought: Improving Robot Readability with Animation Principles. In Proceedings of the 6th International Conference on Human-Robot Interaction(HRI ’11). Association for Computing Machinery, New York, NY, USA, 69–76. http://dx.doi.org/10.1145/1957656.1957674

Digital Library

[101]

Theresa Jean Tanenbaum, Nazely Hartoonian, and Jeffrey Bryan. 2020. ”How Do I Make This Thing Smile?”: An Inventory of Expressive Nonverbal Communication in Commercial Social Virtual Reality Platforms. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. http://dx.doi.org/10.1145/3313831.3376606

Digital Library

[102]

Frank Thomas and Ollie Johnston. 1995. The Illusion of Life: Disney Animation. Disney Editions.

[103]

Güliz Tokadlı, Kaitlyn Ouverson, Chase Meusel, Austin Garcia, Stephen B Gilbert, and Michael C Dorneich. 2018. An Analysis of Video Games Using the Dimensions of Human-Agent Interaction. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 62. SAGE Publications Sage CA: Los Angeles, CA, 716–720.

[104]

Phoebe O. Toups Dugas, Jessica Hammer, William A. Hamilton, Ahmad Jarrah, William Graves, and Oliver Garretson. 2014. A Framework for Cooperative Communication Game Mechanics from Grounded Theory. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play(CHI PLAY ’14). ACM, New York, NY, USA, 257–266. http://dx.doi.org/10.1145/2658537.2658681

Digital Library

[105]

Phoebe O. Toups Dugas and Andruid Kerne. 2007. Implicit Coordination in Firefighting Practice: Design Implications for Teaching Fire Emergency Responders. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’07). Association for Computing Machinery, New York, NY, USA, 707–716. http://dx.doi.org/10.1145/1240624.1240734

Digital Library

[106]

Phoebe O. Toups Dugas, Nicolas Lalone, Sultan A. Alharthi, Hitesh Nidhi Sharma, and Andrew M. Webb. 2019. Making Maps Available for Play: Analyzing the Design of Game Cartography Interfaces. ACM Trans. Comput.-Hum. Interact. 26, 5 (July 2019). http://dx.doi.org/10.1145/3336144

Digital Library

[107]

Phoebe O. Toups Dugas, Nicolas LaLone, Katta Spiel, and Bill Hamilton. 2020. Paper to Pixels: A Chronicle of Map Interfaces in Games. In Proceedings of the 2020 ACM Designing Interactive Systems Conference(DIS ’20). Association for Computing Machinery, New York, NY, USA, 1433–1451. http://dx.doi.org/10.1145/3357236.3395502

Digital Library

[108]

Roel Vertegaal, Robert Slagter, Gerrit van der Veer, and Anton Nijholt. 2001. Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’01. ACM Press, Seattle, Washington, United States, 301–308. http://dx.doi.org/10.1145/365024.365119

Digital Library

[109]

Michael Walker, Hooman Hedayati, Jennifer Lee, and Daniel Szafir. 2018. Communicating robot motion intent with augmented reality. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 316–324.

Digital Library

[110]

Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–15. http://dx.doi.org/10.1145/3290605.3300831

Digital Library

[111]

Robert H. Wortham and Andreas Theodorou. 2017. Robot transparency, trust and utility. Connection Science 29, 3 (July 2017), 242–248. http://dx.doi.org/10.1080/09540091.2017.1313816 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/09540091.2017.1313816.

Digital Library

[112]

Julia L. Wright, Jessie Y. C. Chen, and Shan G. Lakhmani. 2020. Agent Transparency and Reliability in Human–Robot Interaction: The Influence on User Confidence and Perceived Reliability. IEEE Transactions on Human-Machine Systems 50, 3 (June 2020), 254–263. http://dx.doi.org/10.1109/THMS.2019.2925717 Conference Name: IEEE Transactions on Human-Machine Systems.

[113]

Jason Wuertz, Sultan A. Alharthi, William A. Hamilton, Scott Bateman, Carl Gutwin, Anthony Tang, Phoebe O. Toups Dugas, and Jessica Hammer. 2018. A Design Framework for Awareness Cues in Distributed Multiplayer Games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1–14. http://dx.doi.org/10.1145/3173574.3173817

Digital Library

[114]

Greg Zacharias. 2019. Autonomous horizons: the way forward. Air University Press ; Curtis E. LeMay Center for Doctrine Development and Education, Maxwell Air Force Base, Alabama.

Cited By

Neumann AWelten SKunz JKlamma RDecker S(2023)HAICO: A Monitoring and Transaction Chatbot for Distributed Analytics Workflows2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00092(707-709)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00092

Index Terms

“I See You!”: A Design Framework for Interface Cues about Agent Visual Perception from a Thematic Analysis of Videogames
1. General and reference
  1. Cross-computing tools and techniques
    1. Design
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
  2. Ubiquitous and mobile computing

Recommendations

[Hidden] / [Caution] / [Danger]: How Video Games Can Inform the Design of Sight Cues for Agents
HAI '21: Proceedings of the 9th International Conference on Human-Agent Interaction

As artificial agents proliferate into society, there will be more and more situations in which they need to communicate their capabilities to humans, including what they can “see.” Humans do this with each other by using mental models of human ...
Read More
Time perception, immersion and music in videogames
BCS '10: Proceedings of the 24th BCS Interaction Specialist Group Conference

People who play videogames often report the sense of immersion in the game with a particular feature of immersion being a loss of the sense of time passing. In this paper, we investigate if altering the degree of immersion in a videogame really does ...
Read More
A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency

With the emerging development of three-dimensional (3D) related technologies, 3D visual saliency modeling is becoming particularly important and challenging. This paper presents a new depth perception and visual comfort guided saliency computational ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
945
Total Downloads

Downloads (Last 12 months)350
Downloads (Last 6 weeks)17

Other Metrics

View Author Metrics

Citations

Cited By

Neumann AWelten SKunz JKlamma RDecker S(2023)HAICO: A Monitoring and Transaction Chatbot for Distributed Analytics Workflows2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00092(707-709)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00092

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents