research-article

”And then they died”: Using Action Sequences for Data Driven, Context Aware Gameplay Analysis

Authors:

Erica Kleinman,

Truong-Huy D. Nguyen,

Casper Harteveld,

Magy Seif El-NasrAuthors Info & Claims

FDG '20: Proceedings of the 15th International Conference on the Foundations of Digital Games

Article No.: 63, Pages 1 - 12

https://doi.org/10.1145/3402942.3402962

Published: 17 September 2020 Publication History

Abstract

Many successful games rely heavily on data analytics to understand players and inform design. Popular methodologies focus on machine learning and statistical analysis of aggregated data. While effective in extracting information regarding player action, much of the context regarding when and how those actions occurred is lost. Qualitative methods allow researchers to examine context and derive meaningful explanations about the goals and motivations behind player behavior, but are difficult to scale. In this paper, we build on previous work by combining two existing methodologies: Interactive Behavior Analytics (IBA) [2] and sequence analysis (SA), in order to create a novel, mixed methods, human-in-the-loop data analysis methodology that uses behavioral labels and visualizations to allow analysts to examine player behavior in a way that is context sensitive, scalable, and generalizable. We present the methodology along with a case study demonstrating how it can be used to analyze behavioral patterns of teamwork in the popular multiplayer game Defense of the Ancients 2 (DotA 2).

References

[1]

Andrew Abbott. 1995. Sequence analysis: new methods for old ideas. Annual review of sociology 21, 1 (1995), 93–113.

[2]

Sabbir Ahmad, Andy Bryant, Erica Kleinman, Zhaoqing Teng, Truong-Huy D Nguyen, and Magy Seif El-Nasr. 2019. Modeling Individual and Team Behavior through Spatio-temporal Analysis. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM, 601–612.

Digital Library

[3]

Shaowen Bardzell, Jeffrey Bardzell, Tyler Pace, and Kayce Reed. 2008. Blissfully productive: grouping and cooperation in world of warcraft instance runs. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 357–360.

Digital Library

[4]

Francis Bisson, Hugo Larochelle, and Froduald Kabanza. 2015. Using a recursive neural network to learn an agent’s decision model for plan recognition. In Twenty-Fourth International Joint Conference on Artificial Intelligence.

[5]

Blizzard Entertainment. 1999. StarCraft. [Digital Game].

[6]

Blizzard Entertainment. 2004. World of Warcraft. [Digital Game].

[7]

Guillaume Bosc, Mehdi Kaytoue, Chedy Raïssi, and Jean-François Boulicaut. 2013. Strategic Patterns Discovery in RTS-games for E-Sport with Sequential Pattern Mining. In MLSA@ PKDD/ECML. 11–20.

[8]

Pablo Bosch. 2018. Predicting the winner of NFL-games using Machine and Deep Learning.(2018).

[9]

Bungie. 2014. Destiny. [Digital Game].

[10]

Alessandro Canossa, Anders Drachen, and Janus Rau Møller Sørensen. 2011. Arrrgghh!!!: blending quantitative and qualitative methods to detect player frustration. In Proceedings of the 6th international conference on foundations of digital games. ACM, 61–68.

Digital Library

[11]

Alessandro Canossa, Sasha Makarovych, Julian Togelius, and Anders Drachenn. 2018. Like a DNA string: Sequence-based player profiling in Tom Clancy’s the Division. In Fourteenth Artificial Intelligence and Interactive Digital Entertainment Conference.

[12]

Olivier Cavadenti, Victor Codocedo, Jean-François Boulicaut, and Mehdi Kaytoue. 2016. What did i do wrong in my MOBA game? Mining patterns discriminating deviant behaviours. In Data Science and Advanced Analytics (DSAA), 2016 IEEE International Conference on. IEEE, 662–671.

[13]

Zhengxing Chen, Magy Seif El Nasr, Alessandro Canossa, Jeremy Badler, Stefanie Tignor, and Randy Colvin. 2015. Modeling individual differences through frequent pattern mining on role-playing game actions. In Eleventh Artificial Intelligence and Interactive Digital Entertainment Conference.

[14]

Anders Drachen, Rafet Sifa, Christian Bauckhage, and Christian Thurau. 2012. Guns, swords and data: Clustering of player behavior in computer games in the wild. In 2012 IEEE conference on Computational Intelligence and Games (CIG). IEEE, 163–170.

[15]

Anders Drachen, Matthew Yancey, John Maguire, Derrek Chu, Iris Yuhui Wang, Tobias Mahlmann, Matthias Schubert, and Diego Klabajan. 2014. Skill-based differences in spatio-temporal team behaviour in defence of the ancients 2 (dota 2). In 2014 IEEE Games Media Entertainment. IEEE, 1–8.

[16]

Joshua A Eaton, David J Mendonça, and Matthew-Donald D Sangster. 2018. Attack, Damage and Carry: Role Familiarity and Team Performance in League of Legends. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 62. SAGE Publications Sage CA: Los Angeles, CA, 130–134.

[17]

Joshua A Eaton, Matthew-Donald D Sangster, Molly Renaud, David J Mendonca, and Wayne D Gray. 2017. Carrying the Team: The Importance of One Player’s Survival for Team Success in League of Legends. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 61. SAGE Publications Sage CA: Los Angeles, CA, 272–276.

[18]

Christoph Eggert, Marc Herrlich, Jan Smeddinck, and Rainer Malaka. 2015. Classification of player roles in the team-based multi-player game dota 2. In International Conference on Entertainment Computing. Springer, 112–125.

Digital Library

[19]

Simon Ferrari. 2013. From Generative to Conventional Play: MOBA and League of Legends. In DiGRA Conference.

[20]

Guo Freeman and Donghee Yvette Wohn. 2019. Understanding eSports team formation and coordination. Computer supported cooperative work (CSCW) 28, 1-2 (2019), 95–126.

[21]

Alexis Gabadinho, Gilbert Ritschard, Nicolas Séverin Mueller, and Matthias Studer. 2011. Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software 40, 4 (2011), 1–37.

[22]

Nour Halabi, Günter Wallner, and Pejman Mirza-Babaei. 2019. Assessing the impact of visual design on the interpretation of aggregated playtesting data visualization. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 639–650.

Digital Library

[23]

Danial Hooshyar, Moslem Yousefi, and Heuiseok Lim. 2018. Data-driven approaches to game player modeling: a systematic literature review. ACM Computing Surveys (CSUR) 50, 6 (2018), 90.

Digital Library

[24]

Britton Horn, Amy K Hoover, Jackie Barnes, Yetunde Folajimi, Gillian Smith, and Casper Harteveld. 2016. Opening the black box of play: Strategy analysis of an educational game. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play. ACM, 142–153.

Digital Library

[25]

Zaheer Hussain and Mark D Griffiths. 2009. The attitudes, feelings, and experiences of online gamers: a qualitative analysis. CyberPsychology & Behavior 12, 6 (2009), 747–753.

[26]

Sahoon H Kim and Michael K Thomas. 2015. A stage theory model of professional video game players in South Korea: The socio-cultural dimensions of the development of expertise. Asian Journal of Information Technology 14, 5 (2015), 176–186.

[27]

Nicholas Kinkade, L Jolla, and K Lim. 2015. Dota 2 win prediction. Univ Calif 1(2015), 1–13.

[28]

Yubo Kou and Xinning Gui. 2014. Playing with strangers: understanding temporary teams in league of legends. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play. ACM, 161–169.

Digital Library

[29]

Yong Ming Kow and Timothy Young. 2013. Media technologies and learning in the starcraft esport community. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 387–398.

Digital Library

[30]

Choong-Soo Lee and Ivan Ramler. 2017. Identifying and evaluating successful non-meta strategies in league of legends. In Proceedings of the 12th International Conference on the Foundations of Digital Games. ACM, 1.

Digital Library

[31]

Timothy C Lisk, Ugur T Kaplancali, and Ronald E Riggio. 2012. Leadership in multiplayer online gaming environments. Simulation & Gaming 43, 1 (2012), 133–149.

Digital Library

[32]

Ilya Makarov, Dmitry Savostyanov, Boris Litvyakov, and Dmitry I Ignatov. 2017. Predicting Winning Team and Probabilistic Ratings in “Dota 2” and “Counter-Strike: Global Offensive” Video Games. In International Conference on Analysis of Images, Social Networks and Texts. Springer, 183–196.

[33]

Wookhee Min, Alok Baikadi, Bradford Mott, Jonathan Rowe, Barry Liu, Eun Young Ha, and James Lester. 2016. A generalized multidimensional evaluation framework for player goal recognition. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.

Digital Library

[34]

Wookhee Min, Bradford Mott, Jonathan Rowe, Robert Taylor, Eric Wiebe, Kristy Elizabeth Boyer, and James Lester. 2017. Multimodal goal recognition in open-world digital games. In Thirteenth Artificial Intelligence and Interactive Digital Entertainment Conference.

Digital Library

[35]

Wookhee Min, Bradford W Mott, Jonathan P Rowe, Barry Liu, and James C Lester. 2016. Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks. In IJCAI. 2590–2596.

[36]

Pejman Mirza-Babaei, Sebastian Long, Emma Foley, and Graham McAllister. 2011. Understanding the Contribution of Biometrics to Games User Research. In DiGRA conference. Citeseer.

[37]

Dinara Moura, Magy Seif el Nasr, and Christopher D Shaw. 2011. Visualizing and understanding players’ behavior in video games: discovering patterns and supporting aggregation and comparison. In Proceedings of the 2011 ACM SIGGRAPH symposium on video games. ACM, 11–15.

Digital Library

[38]

Ilya Musabirov, Denis Bulygin, Paul Okopny, and Ksenia Konstantinova. 2018. Event-driven Spectators’ Communication in Massive eSports Online Chats. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW564.

Digital Library

[39]

Bonnie Nardi and Justin Harris. 2006. Strangers and friends: Collaborative play in World of Warcraft. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work. ACM, 149–158.

Digital Library

[40]

Fernando Felix do Nascimento Junior, Allan Sales da Costa Melo, Igor Barbosa da Costa, and Leandro Balby Marinho. 2017. Profiling successful team behaviors in League of Legends. In Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web. ACM, 261–268.

Digital Library

[41]

NCSOFT. 1998. Lineage. [Digital Game].

[42]

Truong-Huy D Nguyen, Magy Seif El-Nasr, and Alessandro Canossa. 2015. Glyph: Visualization Tool for Understanding Problem Solving Strategies in Puzzle Games. In FDG.

[43]

Hao Yi Ong, Sunil Deolalikar, and Mark Peng. 2015. Player Behavior and Optimal Team Composition for Online Multiplayer Games. arXiv preprint arXiv:1503.02230(2015).

[44]

Nataliia Pobiedina, Julia Neidhardt, Maria del Carmen Calatrava Moreno, Laszlo Grad-Gyenge, and Hannes Werthner. 2013. On successful team formation: Statistical analysis of a multiplayer online game. In 2013 IEEE 15th Conference on Business Informatics. IEEE, 55–62.

Digital Library

[45]

Patrick Prax. 2010. Leadership style in World of Warcraft raid guilds. Proceedings of DiGRA Nordic(2010).

[46]

Daniel Ramirez-Cano, Simon Colton, and Robin Baumgarten. 2010. Player classification using a meta-clustering approach. In Proceedings of the 3rd Annual International Conference Computer Games, Multimedia & Allied Technology. 297–304.

[47]

Reuters. 2019. Global esports revenues to top 1 billion in 2019: report. shorturl.at/mxyIW

[48]

Riot Games. 2009. League of Legends. [Digital Game].

[49]

Matthias Studer Gilbert Ritschard and Matthias Studer. 2018. Sequence Analysis and Related Approaches. Springer.

[50]

Alain Saas, Anna Guitart, and Africa Periánez. 2016. Discovering playing patterns: Time series clustering of free-to-play game data. In 2016 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, 1–8.

Digital Library

[51]

Matthias Schubert, Anders Drachen, and Tobias Mahlmann. 2016. Esports analytics through encounter detection. In Proceedings of the MIT Sloan Sports Analytics Conference, Vol. 1. 2016.

[52]

Aleksandr Semenov, Peter Romov, Sergey Korolev, Daniil Yashkov, and Kirill Neklyudov. 2016. Performance of machine learning algorithms in predicting game outcome from drafts in dota 2. In International Conference on Analysis of Images, Social Networks and Texts. Springer, 26–37.

[53]

Geoff Skinner and Toby Walmsley. 2019. Artificial Intelligence and Deep Learning in Video Games A Brief Review. In 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS). IEEE, 404–408.

[54]

Tom Stafford, Sam Devlin, Rafet Sifa, and Anders Drachen. 2017. Exploration and skill acquisition in a major online game. In The 39th Annual Meeting of the Cognitive Science Society (CogSci). York.

[55]

Constance A Steinkuehler. 2004. Learning in massively multiplayer online games. In Proceedings of the 6th international conference on Learning sciences. International Society of the Learning Sciences, 521–528.

[56]

Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press.

Digital Library

[57]

Adam Summerville, Michael Cook, and Ben Steenhuisen. 2016. Draft-analysis of the Ancients: predicting draft picks in DotA 2 using machine learning. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.

[58]

Josep Valls-Vargas, Santiago Ontanón, and Jichen Zhu. 2015. Exploring player trace segmentation for dynamic play style prediction. In Eleventh Artificial Intelligence and Interactive Digital Entertainment Conference.

[59]

Valve Corporation. 2013. Defense of the Ancients 2. [Digital Game].

[60]

Wesley van den Broek, Günter Wallner, and Regina Bernhaupt. 2019. Modata–Improving Dota 2 Experience and Spectatorship through Tangible Gameplay Visualization. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 723–730.

Digital Library

[61]

Günter Wallner, Nour Halabi, and Pejman Mirza-Babaei. 2019. Aggregated visualization of playtesting data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[62]

Günter Wallner and Simone Kriglstein. 2012. A spatiotemporal visualization approach for the analysis of gameplay data. In Proceedings of the SIGCHI conference on human factors in computing systems. 1115–1124.

Digital Library

[63]

Günter Wallner and Simone Kriglstein. 2013. Visualization-based analysis of gameplay data–a review of literature. Entertainment Computing 4, 3 (2013), 143–155.

[64]

Pu Yang, Brent E Harrison, and David L Roberts. 2014. Identifying patterns in combat that are predictive of success in MOBA games. In FDG.

[65]

Yifan Yang, Tian Qin, and Yu-Heng Lei. 2016. Real-time esports match result prediction. arXiv preprint arXiv:1701.03162(2016).

[66]

Veronica Zammitto, Pejman Mirza-Babaei, Ian Livingston, Marina Kobayashi, and Lennart E Nacke. 2014. Player experience: mixed methods and reporting results. In CHI’14 Extended Abstracts on Human Factors in Computing Systems. ACM, 147–150.

Cited By

Thurler LMelo SMurta LKohwalter TClua E(2025)Using provenance and replay for qualitative analysis of gameplay sessionsEntertainment Computing10.1016/j.entcom.2024.10077852(100778)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100778
Costa Kohwalter TGresta Paulino Murta LWalter Gonzalez Clua E(2025)Prov-DIFF: Play traces analysis through provenance differencesEntertainment Computing10.1016/j.entcom.2024.10077752(100777)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100777
Mane SElmqvist N(2024)"Wichita 1-1, Fox Three" - The Role of 3D Telemetry Analysis in Combat Flight SimulationProceedings of the ACM on Human-Computer Interaction10.1145/36771148:CHI PLAY(1-27)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677114
Show More Cited By

”And then they died”: Using Action Sequences for Data Driven, Context Aware Gameplay Analysis

Recommendations

Data-Driven Game Development: Ethical Considerations
FDG '20: Proceedings of the 15th International Conference on the Foundations of Digital Games

In recent years, the games industry has made a major move towards data-driven development, using data analytics and player modeling to inform design decisions. Data-driven techniques are beneficial as they allow for the study of player behavior at scale,...
Blending immersive gameplay with intense exercise using asynchronous exergaming
GAS '16: Proceedings of the 5th International Workshop on Games and Software Engineering

Exergames are video games that encourage players to do physical exercises (e.g. yoga, aerobics). While exergames provide some motivational and fitness benefits, they are limited by shallow gameplay and low-intensity exercise. This occurs because, in ...
Profiling Personality Traits with Games
Special Issue on Highlights of ACM IUI 2017

Trying to understand a player's characteristics with regards to a computer game is a major line of research known as player modeling. The purpose of player modeling is typically the adaptation of the game itself. We present two studies that extend ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

FDG '20: Proceedings of the 15th International Conference on the Foundations of Digital Games

September 2020

804 pages

ISBN:9781450388078

DOI:10.1145/3402942

Editors:
Georgios N. Yannakakis
University of Malta
,
Antonios Liapis
University of Malta
,
Penny Kyburz
Australian National University
,
Vanessa Volz
Modl.AI
,
Foaad Khosmood
California Polytechnic State University
,
Phil Lopes
École Polytechnique Fédèrale de Lausanne

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

FDG '20

FDG '20: International Conference on the Foundations of Digital Games

September 15 - 18, 2020

Bugibba, Malta

Acceptance Rates

Overall Acceptance Rate 152 of 415 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
350
Total Downloads

Downloads (Last 12 months)77
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Thurler LMelo SMurta LKohwalter TClua E(2025)Using provenance and replay for qualitative analysis of gameplay sessionsEntertainment Computing10.1016/j.entcom.2024.10077852(100778)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100778
Costa Kohwalter TGresta Paulino Murta LWalter Gonzalez Clua E(2025)Prov-DIFF: Play traces analysis through provenance differencesEntertainment Computing10.1016/j.entcom.2024.10077752(100777)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100777
Mane SElmqvist N(2024)"Wichita 1-1, Fox Three" - The Role of 3D Telemetry Analysis in Combat Flight SimulationProceedings of the ACM on Human-Computer Interaction10.1145/36771148:CHI PLAY(1-27)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677114
Kleinman EXu JPfau JSeif El-Nasr M(2024)"Trust the Process": An Exploratory Study of Process Visualizations for Self-Reflection in League of LegendsProceedings of the ACM on Human-Computer Interaction10.1145/36771118:CHI PLAY(1-28)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677111
Pfau JSeif El-Nasr M(2024)On Video Game Balancing: Joining Player- and Data-Driven AnalyticsGames: Research and Practice10.1145/36758072:3(1-30)Online publication date: 30-Aug-2024
https://dl.acm.org/doi/10.1145/3675807
Kleinman ESeif El-Nasr MPfau JKriglstein SWallner GMelhart DYannakakis GZhu JWatson BHarteveld C(2024)Ethics and Transparency in Game DataCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678859(466-470)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678859
Pfau JCharan MKleinman ESeif El-Nasr M(2024)Damage Optimization in Video Games: A Player-Driven Co-Creative ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642643(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642643
Wu YWang MChu WWang G(2023)Organization Preference Knowledge Acquisition of Multi-Platform Aircraft Mission System Utilizing Frequent Closed Itemset MiningAerospace10.3390/aerospace1002016610:2(166)Online publication date: 10-Feb-2023
https://doi.org/10.3390/aerospace10020166
Margetis GApostolakis KNtoa SChatzakis ISykianaki EMarkopoulos IStephanidis C(2023)Visual Summarisations for Computer-Assisted Live Color Casting and Direction in League of LegendsHCI in Games10.1007/978-3-031-35930-9_10(133-153)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35930-9_10
Jiang YMin TFan STao RCai W(2022)Towards Understanding Player Behavior in Blockchain Games: A Case Study of AavegotchiProceedings of the 17th International Conference on the Foundations of Digital Games10.1145/3555858.3555883(1-12)Online publication date: 5-Sep-2022
https://dl.acm.org/doi/10.1145/3555858.3555883
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten