Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1111/cgf.15173acmconferencesArticle/Chapter ViewAbstractPublication PagesscaConference Proceedingsconference-collections
research-article

Learning to Move Like Professional Counter-Strike Players

Published: 18 September 2024 Publication History

Abstract

In multiplayer, first-person shooter games like Counter-Strike: Global Offensive (CS:GO), coordinated movement is a critical component of high-level strategic play. However, the complexity of team coordination and the variety of conditions present in popular game maps make it impractical to author hand-crafted movement policies for every scenario. We show that it is possible to take a data-driven approach to creating human-like movement controllers for CS:GO. We curate a team movement dataset comprising 123 hours of professional game play traces, and use this dataset to train a transformer-based movement model that generates human-like team movement for all players in a "Retakes" round of the game. Importantly, the movement prediction model is efficient. Performing inference for all players takes less than 0.5 ms per game step (amortized cost) on a single CPU core, making it plausible for use in commercial games today. Human evaluators assess that our model behaves more like humans than both commercially-available bots and procedural movement controllers scripted by experts (16% to 59% higher by TrueSkill rating of "human-like"). Using experiments involving in-game bot vs. bot self-play, we demonstrate that our model performs simple forms of teamwork, makes fewer common movement mistakes, and yields movement distributions, player lifetimes, and kill locations similar to those observed in professional CS:GO match play.

References

[1]
[AAG23] Aydemir, Gorkay, Akan, Adil Kaan, and Güney, Fatma. "Adapt: Efficient multi-agent trajectory prediction with adaptation". Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, 8295--8305 3, 4.
[2]
[BBC*19] Berner, Christopher, Brockman, Greg, Chan, Brooke, et al. "Dota 2 with large scale deep reinforcement learning". arXiv preprint arXiv:1912.06680 (2019) 2, 3.
[3]
[BSK20] Bargteil, Adam W, Shinar, Tamar, and Kry, Paul G. "An introduction to physics-based animation". SIGGRAPH Asia 2020 Courses. 2020, 1--57 6.
[4]
[Cha24] Charts, Steam. Counter-Strike 2 - Steam Charts. Jan. 2024. url: https://steamcharts.com/app/730 2.
[5]
[Cor19] Corporation, NVIDIA. NVIDIA T4 Tensor Core GPU for AI Inference | NVIDIA Data Center. 2019. url: https://www.nvidia.com/en-us/data-center/tesla-t4/ (visited on 04/18/2024) 3.
[6]
[CPV*23] Charalambous, Panayiotis, Pettre, Julien, Vassiliades, Vassilis, et al. "GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and Examples". ACM Transactions on Graphics (TOG) 42.4 (2023), 1--15 3.
[7]
[Cri24] Critic, FPS. PureSkill.gg Data Science | PureSkill.gg Docs. 2024. url: https://docs.pureskill.gg/datascience/ (visited on 04/21/2024) 6.
[8]
[CSLG19] Codevilla, Felipe, Santana, Eder, López, Antonio M, and Gaidon, Adrien. "Exploring the limitations of behavior cloning for autonomous driving". Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 9329--9338 5.
[9]
[DAn23] D'Anastasio, Cecilia. How Influencers Helped Riot Games Turn Valorant Into a Gen Z Hit. 2023. url: https://www.bloomberg.com/news/articles/2023-10-12/how-influencers-helped-riot-games-turn-valorant-into-a-gen-z-hit 2.
[10]
[Dev24] Developer, AAA Video Game. private communication. Jan. 2024 2.
[11]
[DJL19] De Haan, Pim, Jayaraman, Dinesh, and Levine, Sergey. "Causal confusion in imitation learning". Advances in Neural Information Processing Systems 32 (2019) 5.
[12]
[EHE*12] Endres, Felix, Hess, Jürgen, Engelhard, Nikolas, et al. "An evaluation of the RGB-D SLAM system". 2012 IEEE international conference on robotics and automation. IEEE. 2012, 1691--1696 2.
[13]
[FBB*22] (FAIR)†, Meta Fundamental AI Research Diplomacy Team, Bakhtin, Anton, Brown, Noam, et al. "Human-level play in the game of Diplomacy by combining language models with strategic reasoning". Science 378.6624 (2022), 1067--1074 2, 3.
[14]
[GFF12] Gamez, David, Fountas, Zafeirios, and Fidjeland, Andreas K. "A neurally controlled computer game avatar with humanlike behavior". IEEE Transactions on Computational Intelligence and AI in Games 5.1 (2012), 1--14 8.
[15]
[GHT*19] Guss, William H, Houghton, Brandon, Topin, Nicholay, et al. "MineRL: a large-scale dataset of minecraft demonstrations". Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 2442--2448 3.
[16]
[Gli95] Glickman, Mark E. "A comprehensive guide to chess ratings". American Chess Journal 3.1 (1995), 59--102 8.
[17]
[Hin09] Hingston, Philip. "A turing test for computer game bots". IEEE Transactions on Computational Intelligence and AI in Games 1.3 (2009), 169--186 7.
[18]
[Hin10] Hingston, Philip. "A new design for a turing test for bots". Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games. IEEE. 2010, 345--350 7.
[19]
[HLT24] HLTV.org. Counter-Strike News & Coverage | HLTV.org. Jan. 2024. url: https://www.hltv.org/ 6.
[20]
[HMG06] Herbrich, Ralf, Minka, Tom, and Graepel, Thore. "TrueSkill: a Bayesian skill rating system". Advances in neural information processing systems 19 (2006) 8.
[21]
[HMZB23] Huang, Chenguang, Mees, Oier, Zeng, Andy, and Burgard, Wolfram. "Visual language maps for robot navigation". 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2023, 10608--10615 2.
[22]
[HT18] Huang, W. and Terzopoulos, D. "Door and Doorway Etiquette for Virtual Humans". IEEE Transactions on Visualization and Computer Graphics (2018). issn: 1077-2626. 2.
[23]
[Isl05] Isla, Damian. GDC 2005 Proceeding: Handling Complexity in the Halo 2 AI. Mar. 2005. url: https://www.gamedeveloper.com/programming/gdc-2005-proceeding-handling-complexity-in-the-i-halo-2-i-ai 2.
[24]
[Isl13] Isla, Damián. "Third Eye Crime: Building a stealth game around occupancy maps". Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. Vol. 9. 1. 2013, 206--206 6.
[25]
[JCD*19] Jaderberg, Max, Czarnecki, Wojciech M, Dunning, Iain, et al. "Human-level performance in 3D multiplayer games with population-based reinforcement learning". Science 364.6443 (2019), 859--865 2, 3.
[26]
[Jus22] Justesen, Niels. AI Summit: Buffing Bots with Imitation Learning. Game Developers Conference. 2022. url: https://www.gdcvault.com/play/1027942/AI-Summit-Buffing-Bots-with 2.
[27]
[KPH20] Kanervisto, Anssi, Pussinen, Joonas, and Hautamäki, Ville. "Benchmarking end-to-end behavioural cloning on video games". 2020 IEEE conference on games (CoG). IEEE. 2020, 558--565 3.
[28]
[KPT*17] Kolouri, Soheil, Park, Se Rim, Thorpe, Matthew, et al. "Optimal mass transport: Signal processing and machine-learning applications". IEEE signal processing magazine 34.4 (2017), 43--59 8.
[29]
[LC17] Lample, Guillaume and Chaplot, Devendra Singh. "Playing FPS games with deep reinforcement learning". Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 31. 1. 2017 2, 3.
[30]
[Lin20] Lind, David. Game Server Performance on 'Tom Clancy's The Division 2'. Game Developers Conference. Aug. 2020. url: https://gdcvault.com/play/1026706/Game-Server-Performance-on-Tom 2.
[31]
[Mor88] Moravec, Hans P. "Sensor fusion in certainty grids for mobile robots". AI magazine 9.2 (1988), 61--61 6.
[32]
[NVC*22] Ngiam, Jiquan, Vasudevan, Vijay, Caine, Benjamin, et al. "Scene transformer: A unified architecture for predicting future trajectories of multiple agents". International Conference on Learning Representations. 2022 2--5, 11.
[33]
[PKL*22] Panayiotou, Andreas, Kyriakou, Theodoros, Lemonari, Marilena, et al. "CCP: Configurable Crowd Profiles". ACM SIGGRAPH 2022 Conference Proceedings. SIGGRAPH '22. Vancouver, BC, Canada: Association for Computing Machinery, 2022. isbn: 9781450393379. url: https://doi.org/10.1145/3528233.3530712 2.
[34]
[POC*23] Park, Joon Sung, O'Brien, Joseph, Cai, Carrie Jun, et al. "Generative agents: Interactive simulacra of human behavior". Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 2023, 1--22 7.
[35]
[PyT24] PyTorch. TorchScript for Deployment - PyTorch Tutorials 2.3.0+cu121 documentation. 2024. url: https://pytorch.org/tutorials/recipes/torchscript_inference.html 6.
[36]
[PZ22] Pearce, Tim and Zhu, Jun. "Counter-strike deathmatch with large-scale behavioural cloning". 2022 IEEE Conference on Games (CoG). IEEE. 2022, 104--111 2, 3, 5, 8.
[37]
[Ran20] Randall, Brent. VALORANT's 128-Ticf Servers | Riot Games Technology. Aug. 2020. url: https://technology.riotgames.com/news/valorants-128-tick-servers 2.
[38]
[Rey87] Reynolds, Craig W. "Flocks, herds and schools: A distributed behavioral model". SIGGRAPH Comput. Graph. 21.4 (Aug. 1987), 25--34. issn: 0097-8930. url: https://doi.org/10.1145/37402.37406 2.
[39]
[SCC*23] Seff, Ari, Cera, Brian, Chen, Dian, et al. "Motionlm: Multi-agent motion forecasting as language modeling". Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, 8579--8590 3, 4.
[40]
[SHS*18] Silver, David, Hubert, Thomas, Schrittwieser, Julian, et al. "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play". Science 362.6419 (2018), 1140--1144 2.
[41]
[SHYK23] Seo, Seokin, Hwang, HyeongJoo, Yang, Hongseok, and Kim, Kee-Eung. "Regularized Behavior Cloning for Blocking the Leakage of Past Action Information". The 37th Conference on Neural Information Processing Systems (NeurIPS 2023). Neural information processing systems foundation. 2023 5.
[42]
[SSS*17] Silver, David, Schrittwieser, Julian, Simonyan, Karen, et al. "Mastering the game of Go without human knowledge". Nature 550.7676 (2017) 3.
[43]
[VSP*17] Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, et al. "Attention is all you need". Advances in neural information processing systems 30 (2017) 4, 5.
[44]
[WLP16] Wolinski, David, Lin, Ming C., and Pettré, Julien. "WarpDriver: context-aware probabilistic motion prediction for crowd simulation". ACM Trans. Graph. 35.6 (Dec. 2016). issn: 0730-0301. url: https://doi.org/10.1145/2980179.2982442 2.
[45]
[XS22] Xenopoulos, Peter and Silva, Claudio. "ESTA: An Esports Trajectory and Action Dataset". arXiv preprint arXiv:2209.09861 (2022) 6.
[46]
[Yin23] Yin-Poole, Wesley. Call of Duty Has 90 Million Players, Half of All Engagement on Mobile. 2023. url: https://www.ign.com/articles/call-of-duty-has-90-million-players-half-of-all-engagement-on-mobile 2.
[47]
[YWOK21] Yuan, Ye, Weng, Xinshuo, Ou, Yanglan, and Kitani, Kris M. "Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting". Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, 9813--9823 4.
[48]
[ZHZ*24] Zheng, Duo, Huang, Shijia, Zhao, Lin, et al. "Towards learning a generalist model for embodied navigation". Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, 13624--13634 2.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SCA '24: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation
August 2024
276 pages

Sponsors

In-Cooperation

  • EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Eurographics Association

Goslar, Germany

Publication History

Published: 18 September 2024

Check for updates

Qualifiers

  • Research-article

Conference

SCA '24
Sponsor:
SCA '24: ACM SIGGRAPH/Eurographics Symposium on Computer Animation
August 21 - 23, 2024
Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 19
    Total Downloads
  • Downloads (Last 12 months)19
  • Downloads (Last 6 weeks)8
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media