research-article

"I Want To See How Smart This AI Really Is": Player Mental Model Development of an Adversarial AI Player

Authors:

Jennifer Villareale,

Casper Harteveld,

Jichen ZhuAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue CHI PLAY

Article No.: 219, Pages 1 - 26

https://doi.org/10.1145/3549482

Published: 31 October 2022 Publication History

Abstract

Understanding players' mental models are crucial for game designers who wish to successfully integrate player-AI interactions into their game. However, game designers face the difficult challenge of anticipating how players model these AI agents during gameplay and how they may change their mental models with experience. In this work, we conduct a qualitative study to examine how a pair of players develop mental models of an adversarial AI player during gameplay in the multiplayer drawing game iNNk. We conducted ten gameplay sessions in which two players (n = 20, 10 pairs) worked together to defeat an AI player. As a result of our analysis, we uncovered two dominant dimensions that describe players' mental model development (i.e., focus and style). The first dimension describes the focus of development which refers to what players pay attention to for the development of their mental model (i.e., top-down vs. bottom-up focus). The second dimension describes the differences in the style of development, which refers to how players integrate new information into their mental model (i.e., systematic vs. reactive style). In our preliminary framework, we further note how players process a change when a discrepancy occurs, which we observed occur through comparisons (i.e., compare to other systems, compare to gameplay, compare to self). We offer these results as a preliminary framework for player mental model development to help game designers anticipate how different players may model adversarial AI players during gameplay.

Supplementary Material

ZIP File (v6chip219aux.zip)

Download
43.21 MB

MP4 File (v6chip219.mp4)

Supplemental video

Download
43.28 MB

References

[1]

David Ackermann, Michael J Tauber, and Michael J Tauber. 1990. Mental models and human-computer interaction 1. Number 3. North Holland.

[2]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al . 2019. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[3]

Craig G Anderson, Jen Dalsen, Vishesh Kumar, Matthew Berland, and Constance Steinkuehler. 2018. Failing up: How failure in a game environment promotes learning through discourse. Thinking Skills and Creativity 30 (2018), 135--144.

[4]

Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in human-ai team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2--11.

[5]

Pascal Béguin and Pierre Rabardel. 2000. Designing for instrument-mediated activity. Scandinavian Journal of Information Systems 12, 1 (2000), 1.

[6]

Hao-Fei Cheng, Ruotong Wang, Zheng Zhang, Fiona O'Connell, Terrance Gray, F Maxwell Harper, and Haiyi Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the 2019 chi conference on human factors in computing systems. 1--12.

Digital Library

[7]

Michael Chromik, Malin Eiband, Felicitas Buchner, Adrian Krüger, and Andreas Butz. 2021. I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI. In 26th International Conference on Intelligent User Interfaces. 307--317.

Digital Library

[8]

Gabriele Cimolino, Sam Lee, Quentin Petraroia, and TC Nicholas Graham. 2019. Oui, Chef!!: Supervised Learning for Novel Gameplay with Believable AI. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 241--246.

Digital Library

[9]

Allan Collins and Dedre Gentner. 1987. How people construct mental models. Cultural models in language and thought 243 (1987), 243--265.

[10]

Michael Cook, Mirjam Eladhari, Andy Nealen, Mike Treanor, Eddy Boxerman, Alex Jaffe, Paul Sottosanti, and Steve Swink. 2016. PCG-Based Game Design Patterns. arXiv preprint arXiv:1610.03138 (2016).

[11]

Kenneth James Williams Craik. 1943. The nature of explanation. Vol. 445. CUP Archive.

[12]

Mihaly Csikszentmihalyi. 1990. Flow: The psychology of optimal experience. Vol. 1990. Harper & Row New York.

[13]

James K Doyle and David N Ford. 1998. Mental models concepts for system dynamics research. System dynamics review: the journal of the System Dynamics Society 14, 1 (1998), 3--29.

[14]

Kate Ehrlich. 1996. Applied mental models in human--computer interaction. Mental models in cognitive science: Essays in honour of Phil Johnson-Laird (1996), 223.

[15]

Katrin Etzrodt and Sven Engesser. 2021. Voice-based agents as personified things: Assimilation and accommodation as equilibration of doubt. Human-Machine Communication 2 (2021), 57--76.

[16]

Creso Franco and Dominique Colinvaux. 2000. Grasping mental models. In Developing models in science education. Springer, 93--118.

[17]

Caleb S Furlough and Douglas J Gillan. 2018. Mental models: structural differences and the role of experience. Journal of Cognitive Engineering and Decision Making 12, 4 (2018), 269--287.

[18]

James Paul Gee. 2005. Learning by design: Good video games as learning machines. E-learning and Digital Media 2, 1 (2005), 5--16.

[19]

Dedre Gentner. 2002. Mental models, psychology of. In International encyclopedia of the social and behavioral sciences. Elsevier Science, 9683--9687.

[20]

Dedre Gentner and Albert L. Stevens. 1983. Mental models. Lawrence Erlbaum Associates.

[21]

Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, Maria Ruiz, Sarah Miller, David R Millen, Murray Campbell, et al . 2020. Mental models of AI agents in a cooperative game setting. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[22]

Google. 2016. Quick, Draw! https://quickdraw.withgoogle.com/

[23]

John Graham, Liya Zheng, and Cleotilde Gonzalez. 2006. A Cognitive Approach to Game Usability and Design: Mental Model Development in Novice Real-Time Strategy Gamers. CyberPsychology & Behavior 9, 3 (2006), 361--366.

[24]

Robert C Gray, Jichen Zhu, and Santiago Ontañón. 2021. Multiplayer Modeling via Multi-Armed Bandits. In 2021 IEEE Conference on Games (CoG). IEEE, 01--08.

Digital Library

[25]

David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).

[26]

Clara E Hill, Sarah Knox, Barbara J Thompson, Elizabeth Nutt Williams, Shirley A Hess, and Nicholas Ladany. 2005. Consensual qualitative research: An update. Journal of counseling psychology 52, 2 (2005), 196.

[27]

Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 (2018).

[28]

Angel Hsing-Chi Hwang and Andrea Stevenson Won. 2022. AI in Your Mind: Counterbalancing Perceived Agency and Experience in Human-AI Interaction. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--10.

[29]

Daniel Jallov, Sebastian Risi, and Julian Togelius. 2016. EvoCommander: A novel game based on evolving and switching between artificial brains. IEEE Transactions on Computational Intelligence and AI in Games 9, 2 (2016), 181--191.

[30]

Bernard Jansen, Joni Salminen, Soon-gyo Jung, and Kathleen Guan. 2021. Data-driven personas. Synthesis Lectures on Human-Centered Informatics 14, 1 (2021), i--317.

[31]

Philip Nicholas Johnson-Laird. 1983. Mental models: Towards a cognitive science of language, inference, and consciousness. Number 6. Harvard University Press.

[32]

Natalie A Jones, Helen Ross, Timothy Lynam, Pascal Perez, and Anne Leitch. 2011. Mental models: an interdisciplinary synthesis of theory and methods. Ecology and Society 16, 1 (2011).

[33]

Jesper Juul. 2013. The art of failure: An essay on the pain of playing video games. MIT press.

[34]

Manu Kapur and Nikol Rummel. 2012. Productive failure in learning from generation and invention activities. Instructional Science 40, 4 (2012), 645--650.

[35]

Cecilia Katzeff. 1988. The effect of different conceptual models upon reasoning in a database query writing task. International Journal of Man-Machine Studies 29, 1 (1988), 37--62.

Digital Library

[36]

Cecilia Katzeff. 1990. System demands on mental models for a fulltext database. International Journal of Man-Machine Studies 32, 5 (1990), 483--509.

Digital Library

[37]

Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more? The effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1--10.

Digital Library

[38]

Todd Kulesza, Simone Stumpf, Margaret Burnett, Weng-Keen Wong, Yann Riche, Travis Moore, Ian Oberst, Amber Shinsel, and Kevin McIntosh. 2010. Explanatory debugging: Supporting end-user debugging of machine-learned programs. In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing. IEEE, 41--48.

Digital Library

[39]

Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, and Weng-Keen Wong. 2013. Too much, too little, or just right? Ways explanations impact end users' mental models. In 2013 IEEE Symposium on visual languages and human centric computing. IEEE, 3--10.

[40]

Franco Landriscina. 2013. Simulation and learning. Springer.

[41]

Andreas Lieberoth. 2015. Shallow gamification: Testing psychological effects of framing an activity as a game. Games and Culture 10, 3 (2015), 229--248.

[42]

Xiaodong Lin, Cindy Hmelo, Charles K Kinzer, and Teresa J Secules. 1999. Designing technology to support reflection. Educational technology research and Development 47, 3 (1999), 43--62.

[43]

Mathias Löwe, Jennifer Villareale, Evan Freed, Aleksanteri Sladek, Jichen Zhu, and Sebastian Risi. 2021. Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning. In The 16th International Conference on the Foundations of Digital Games (FDG) 2021. 1--10.

Digital Library

[44]

Tim Marsh. 2016. Slow serious games, interactions and play: Designing for positive and serious experience and reflection. Entertainment computing 14 (2016), 45--53.

[45]

Richard E Mayer, Amanda Mathias, and Karen Wetzell. 2002. Fostering understanding of multimedia messages through pre-training: Evidence for a two-stage theory of mental model construction. Journal of Experimental Psychology: Applied 8, 3 (2002), 147.

[46]

Elisa D Mekler, Ioanna Iacovides, and Julia Ayumi Bopp. 2018. " A Game that Makes You Question..." Exploring the Role of Reflection for the Player Experience. In Proceedings of the 2018 annual symposium on computer-human interaction in play. 315--327.

Digital Library

[47]

Marvin Minsky. 1974. A framework for representing knowledge.

[48]

Avantika Mohapatra. 2020. Designing for AI: A collaborative framework to bridge the gap between designers and data scientists, and enabling designers to create human-centered AI products and services.

[49]

Donald A Norman. 1983. Some observations on mental models. Mental models 7, 112 (1983), 7--14.

[50]

Donald A Norman and Daniel G Bobrow. 1979. Descriptions: An intermediate stage in memory retrieval. Cognitive Psychology 11, 1 (1979), 107--123.

[51]

Donald A Norman and David E Rumelhart. 1978. Accretion, tuning and restructuring: Three modes of learning. Semantic Factors in Cognition. Hillsdale, NJ: Lawrence Erlbaum (1978), 37--53.

[52]

Stephen J Payne. 2003. Users' mental models: The very ideas. HCI models, theories, and frameworks: Toward a multidisciplinary science (2003), 135--156.

[53]

Sean Penney, Jonathan Dodge, Claudia Hilderbrand, Andrew Anderson, Logan Simpson, and Margaret Burnett. 2018. Toward foraging for understanding of StarCraft agents: An empirical study. In 23rd International Conference on Intelligent User Interfaces. 225--237.

Digital Library

[54]

Jan Piskur, Peter Greve, Julian Togelius, and Sebastian Risi. 2015. Braincrafter: An investigation into human-based neural network engineering. In 2015 IEEE Congress on Evolutionary Computation (CEC). IEEE, 2199--2206.

[55]

K Andrew R Richards and Michael A Hemphill. 2018. A practical guide to collaborative qualitative data analysis. Journal of Teaching in Physical Education 37, 2 (2018), 225--231.

[56]

Sebastian Risi, Joel Lehman, David B D'Ambrosio, Ryan Hall, and Kenneth O Stanley. 2015. Petalz: Search-based procedural content generation for the casual gamer. IEEE Transactions on Computational Intelligence and AI in Games 8, 3 (2015), 244--255.

[57]

Yvonne Rogers, Helen Sharp, and Jenny Preece. 2011. Interaction design: beyond human-computer interaction. John Wiley & Sons.

Digital Library

[58]

D Rumelhart and E Schemata. 1980. The building blocks of cognition. Theoretical issues in reading comprehension: Perspectives from cognitive psychology, linguistics, artificial intelligence, and education (1980).

[59]

David E Rumelhart. 1984. Schemata and the cognitive system. (1984).

[60]

Pamela Ann Savage-Knepshield. 2001. Mental models: Issues in construction, congruency, and cognition. Rutgers The State University of New Jersey-New Brunswick.

[61]

Roger C Schank and Robert P Abelson. 2013. Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Psychology Press.

[62]

Joe Tullio, Anind K Dey, Jason Chalecki, and James Fogarty. 2007. How it works: a field study of non-technical users interacting with an intelligent system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 31--40.

Digital Library

[63]

Josep Valls-Vargas, Jichen Zhu, and Santiago Ontañón. 2017. Graph grammar-based controllable generation of puzzles for a learning game about parallel programming. In Proceedings of the 12th International Conference on the Foundations of Digital Games. 1--10.

Digital Library

[64]

Jennifer Villareale, Ana V Acosta-Ruiz, Samuel Adam Arcaro, Thomas Fox, Evan Freed, Robert C Gray, Mathias Löwe, Panote Nuchprayoon, Aleksanteri Sladek, Rush Weigelt, et al . 2020. iNNk: A Multi-Player Game to Deceive a Neural Network. In Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play. 33--37.

Digital Library

[65]

Jennifer Villareale, Colan F. Biemer, Magy Seif El-Nasr, and Jichen Zhu. 2020. Reflection in Game-Based Learning: A Survey of Programming Games. In International Conference on the Foundations of Digital Games. 1--9.

[66]

Barry J Wadsworth. 1984. Piaget's theory of cognitive and affective development (third edition ed.). Longman Publishing.

[67]

Yvonne Waern. 1985. Learning computerized tasks as related to prior task knowledge. International Journal of Man-Machine Studies 22, 4 (1985), 441--455.

[68]

Yvonne Waern. 1990. On the dynamics of mental models. In Selected papers of the 6th Interdisciplinary Workshop on Informatics and Psychology: Mental Models and Human-Computer Interaction 1. 73--93.

[69]

Michael D. Williams, James D. Hollan, and Albert L. Stevens. 1983. Human Reasoning About a Simple Physical System. In International encyclopedia of the social and behavioral sciences. Lawrence Erlbaum Associates, 131--153.

[70]

John R Wilson and Andrew Rutherford. 1989. Mental models: Theory and application in human factors. Human Factors 31, 6 (1989), 617--634.

Digital Library

[71]

Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-examining whether, why, and how human- AI interaction is uniquely difficult to design. In Proceedings of the 2020 chi conference on human factors in computing systems. 1--13.

Digital Library

[72]

Georgios N Yannakakis and Julian Togelius. 2018. Artificial intelligence and games. Vol. 2. Springer.

Digital Library

[73]

Nick Yee. 2006. Motivations for play in online games. CyberPsychology & behavior 9, 6 (2006), 772--775.

[74]

Nick Yee. 2015. Game Motivation Model. https://quanticfoundry.com/

[75]

Yan Zhang. 2009. The construction of mental models of information-rich web spaces: The development process and the impact of task complexity. (2009).

[76]

Yan Zhang. 2013. The development of users' mental models of MedlinePlus in information searching. Library & information science research 35, 2 (2013), 159--170.

[77]

Jichen Zhu and Santiago Ontañón. 2019. Experience management in multi-player games. In 2019 IEEE Conference on Games (CoG). IEEE, 1--6.

Digital Library

[78]

Jichen Zhu and Santiago Ontañón. 2020. Player-centered AI for automatic game personalization: Open problems. In International Conference on the Foundations of Digital Games. 1--8.

Digital Library

[79]

Jichen Zhu, Jennifer Villareale, Nithesh Javvaji, Sebastian Risi, Mathias Löwe, Rush Weigelt, and Casper Harteveld. 2021. Player-AI Interaction: What Neural Network Games Reveal About AI as Play. In Proceedings of the 2021 ACM Conference on Human Factors in Computing Systems (CHI '21).

Digital Library

Cited By

Sidji MSmith WRogerson M(2024)Human-AI Collaboration in Cooperative Games: A Study of Playing Codenames with an LLM AssistantProceedings of the ACM on Human-Computer Interaction10.1145/36770818:CHI PLAY(1-25)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677081
Sivertsen CSalimbeni GLøvlie ABenford SZhu J(2024)Machine Learning Processes As Sources of Ambiguity: Insights from AI ArtProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642855(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642855
Cimolino GChen RGutwin CGraham T(2023)Automation Confusion: A Grounded Theory of Non-Gamers’ Confusion in Partially Automated Action GamesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581116(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581116
Show More Cited By

Index Terms

"I Want To See How Smart This AI Really Is": Player Mental Model Development of an Adversarial AI Player
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Player's model: criteria for a gameplay profile measure
ICEC'10: Proceedings of the 9th international conference on Entertainment computing

Game designers empirically use psychological and sociological player's model to create the gameplay of their video games. These models are generally implicit and always informal. A formal analysis of the player's model leads to define efficient player ...
Player see, player do

While the benefits of physically immersive video games, or exergames, have witnessed much research attention, less is known about the psychosocial processes that enable exergames to be an effective digital tool. An increasing number of exergames feature ...
No player is ideal: why video game designers cannot ethically ignore players' real-world identities

As video games flourish, designers have a responsibility to treat players and potential players justly. In deontological terms, designers are obliged to treat all of them as having intrinsic worth. Since players are a diverse group, designers must not ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue CHI PLAY

CHI PLAY

October 2022

986 pages

EISSN:2573-0142

DOI:10.1145/3570219

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2022

Published in PACMHCI Volume 6, Issue CHI PLAY

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation (NSF)
Novo Nordisk Foundation Grant

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
418
Total Downloads

Downloads (Last 12 months)173
Downloads (Last 6 weeks)14

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sidji MSmith WRogerson M(2024)Human-AI Collaboration in Cooperative Games: A Study of Playing Codenames with an LLM AssistantProceedings of the ACM on Human-Computer Interaction10.1145/36770818:CHI PLAY(1-25)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677081
Sivertsen CSalimbeni GLøvlie ABenford SZhu J(2024)Machine Learning Processes As Sources of Ambiguity: Insights from AI ArtProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642855(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642855
Cimolino GChen RGutwin CGraham T(2023)Automation Confusion: A Grounded Theory of Non-Gamers’ Confusion in Partially Automated Action GamesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581116(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581116
Yong QMitchell A(2023)From Playing the Story to Gaming the System: Repeat Experiences of a Large Language Model-Based Interactive StoryInteractive Storytelling10.1007/978-3-031-47655-6_24(395-409)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47655-6_24

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents