research-article

Neuroevolution based multi-agent system for micromanagement in real-time strategy games

Authors:

Iuhasz Gabriel,

Viorel Negru, and

Daniela ZaharieAuthors Info & Claims

BCI '12: Proceedings of the Fifth Balkan Conference in Informatics

September 2012

Pages 32 - 39

https://doi.org/10.1145/2371316.2371324

Published: 16 September 2012 Publication History

Abstract

The main goal of this paper is the design of a multi-agent system (MAS) that handles unit micromanagement in real time strategy games and is able to adapt/learn during game play. To achieve this we adopted the rtNEAT approach in order to obtain customized neural network topologies, thus avoiding the generation of too complex architectures. Also by defining internal and external inputs for each agent we managed to create independent agents that are able to cooperate and form teams for their mutual benefit and at the same time eliminate unnecessary communication overhead. The MAS was implemented for the real time strategy game StarCraft using JADE multi-agent platform and BWAPI to ensure the interface with the game. We used as a baseline the in game AI and also tested it against other adapting AI systems in order to compare their performance against our system.

References

[1]

Shantia, A., Begue, E., & Wiering, M. (2007). Connectionist Reinforcement Learning for Intelligent Unit Micro Management in StarCraft. Neural Networks, 1794--1801.

[2]

S. C. Bakkes, P. H. Spronck, and H. J. van den Herik, Opponent modelling for case-based adaptive game AI, Entertainment Computing, vol. 1, no. 1, pp. 27--37, 2009.

[3]

P. Spronck, I. Sprinkhuizen-Kuyper, and E. Postma, Difficulty Scaling of Game AI, in Proceedings of the 5th International Conference on Intelligent Games and Simulation (GAME-ON 2004), 2004, pp. 33--37.

[4]

Evans, Richard (2002). Chapter 11.2: Varieties of Learning. In Rabin, Steve. AI Game Programming Wisdom. Charles River Media. pp. 567--578. ISBN 1-58450-077-8.

[5]

M. van der Heijden, S. Bakkes, and P. Spronck, Dynamic formations in real-time strategy games, in Computational Intelligence and Games, 2008. CIG'08. IEEE Symposium On, december 2008, pp. 47--54.

[6]

Stanley, K. O. (2002). Efficient Reinforcement Learning through Evolving Neural Network Topologies. Evolutionary Computation, 1(2), 3.

Digital Library

[7]

Stanley, K. O., Bryant, B. D., & Miikkulainen, R. (2005). Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation.

Digital Library

[8]

AIIDE 2010 StarCraft Competition -- http://eis.ucsc.edu/StarCraftAICompetition

[9]

Weber, B. G., & Mateas, M. (2011). Building Human-Level AI for Real-Time Strategy Games. 2011 AAAI Fall Symposium Series, 329--336.

[10]

Olesen, J. K., Yannakakis, G. N., & Hallam, J. (2008). Real-time challenge balance in an RTS game using rtNEAT. 2008 IEEE Symposium On Computational Intelligence and Games, 87--94. Ieee.

[11]

Miikkulainen, R., & Stanley, K. O. (2011). Competitive Coevolution through Evolutionary Complexification. Journal of Artificial Intelligence Research, 21(1), 63--100.

Digital Library

[12]

Miikkulainen, R., & Stanley, K. O. (2011). Competitive Coevolution through Evolutionary Complexification. Journal of Artificial Intelligence Research, 21(1), 63--100.

Digital Library

[13]

K. Stanley and R. Miikkulainen, Efficient Reinforcement Learning through Evolving Neural Network Topologies, in Proceedings of the Genetic and Evolutionary Computation Conference, 2002, pp. 569--577.

Digital Library

[14]

Buro, M., & Furtak, T. (2004). RTS games and real-time AI research. Proceedings of the Behavior Representation in, 6370, pp. 63--70.

[15]

M. Chung, M. Buro, and J. Schaeffer, Monte Carlo Planning in RTS Games, in Proceedings of the IEEE Symposium on Computational Intelligence and Games. IEEE Press, 2005, pp. 117--124.

[16]

S. Jang, J. Yoon, and S. Cho, Optimal Strategy Selection of NonPlayer Character on Real Time Strategy Game using a Speciated Evolutionary Algorithm, in Proceedings of the IEEE Symposium on Computational Intelligence and Games. IEEE Press, 2009, pp. 75--79.

Digital Library

[17]

AIIDE 2012 StarCraft Competition -- http://tinyurl.com/cv2e2af

[18]

Overmind winner of AIIDE 2010 StarCraft Competition -- http://tinyurl.com/6lhur6b

[19]

Weber, B. G., Mateas, M., & Jhala, A. (2010). Applying Goal-Driven Autonomy to StarCraft. Artificial Intelligence, (Orkin), 101--106.

[20]

NOVA STARCRAFT AI -- http://nova.wolfwork.com/

[21]

Lewis, J. M., Trinh, P., & Kirsh, D. (2011). A Corpus Analysis of Strategy Video Game Play in Starcraft: Brood War. Cognitive Science, 687--692.

[22]

StarCraft AI script selection -- http://tiny.cc/loeuew

[23]

AIIDE 2011 Competition Results -- http://tiny.cc/ipeuew

Cited By

Barambones JCano-Benito JSánchez-Rivero IImbert RRichoux F(2023)Multiagent Systems on Virtual Games: A Systematic Mapping StudyIEEE Transactions on Games10.1109/TG.2022.321415415:2(134-147)Online publication date: Jun-2023
https://doi.org/10.1109/TG.2022.3214154
Han IKim K(2023)Deep ensemble learning of tactics to control the main force in a real-time strategy gameMultimedia Tools and Applications10.1007/s11042-023-15742-x83:4(12059-12087)Online publication date: 24-Jun-2023
https://doi.org/10.1007/s11042-023-15742-x
Zhou WSubagdja BTan AOng D(2021)Hierarchical control of multi-agent reinforcement learning team in real-time strategy (RTS) gamesExpert Systems with Applications: An International Journal10.1016/j.eswa.2021.115707186:COnline publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1016/j.eswa.2021.115707
Show More Cited By

Index Terms

Neuroevolution based multi-agent system for micromanagement in real-time strategy games
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Artificial Intelligence Techniques on Real-time Strategy Games
CSAI '18: Proceedings of the 2018 2nd International Conference on Computer Science and Artificial Intelligence

Real-time strategy (RTS) games can be seen as simulating real and complex dynamic environments in a limited and small world, posing important challenges for the development of artificial intelligence. Existing applications of artificial intelligence ...
Read More
Real-time neuroevolution in the NERO video game

In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the ...
Read More
Neuroevolution for Micromanagement in the Real-Time Strategy Game Starcraft: Brood War
Proceedings of the 26th Australasian Joint Conference on AI 2013: Advances in Artificial Intelligence - Volume 8272

Real-Time Strategy (RTS) games have become an attractive domain for AI research in recent years, due to their dynamic, multi-agent and multi-objective environments. Micromanagement, a core component of many RTS games, involves the control of multiple ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

BCI '12: Proceedings of the Fifth Balkan Conference in Informatics

September 2012

312 pages

ISBN:9781450312400

DOI:10.1145/2371316

General Chair:
Mirjana Ivanović
Univ. of Novi Sad, Serbia
,
Program Chair:
Zoran Budimac
Univ. of Novi Sad, Serbia

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

MSTD: Ministry of Education, Science and Technological Development - Serbia
Novi Sad: Faculty of Technical Sciences, University of Novi Sad

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

BCI '12

Sponsor:

MSTD
Novi Sad

BCI '12: Balkan Conference in Informatics, 2012

September 16 - 20, 2012

Novi Sad, Serbia

Acceptance Rates

Overall Acceptance Rate 97 of 250 submissions, 39%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
445
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Other Metrics

View Author Metrics

Citations

Cited By

Barambones JCano-Benito JSánchez-Rivero IImbert RRichoux F(2023)Multiagent Systems on Virtual Games: A Systematic Mapping StudyIEEE Transactions on Games10.1109/TG.2022.321415415:2(134-147)Online publication date: Jun-2023
https://doi.org/10.1109/TG.2022.3214154
Han IKim K(2023)Deep ensemble learning of tactics to control the main force in a real-time strategy gameMultimedia Tools and Applications10.1007/s11042-023-15742-x83:4(12059-12087)Online publication date: 24-Jun-2023
https://doi.org/10.1007/s11042-023-15742-x
Zhou WSubagdja BTan AOng D(2021)Hierarchical control of multi-agent reinforcement learning team in real-time strategy (RTS) gamesExpert Systems with Applications: An International Journal10.1016/j.eswa.2021.115707186:COnline publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1016/j.eswa.2021.115707
Fan XWu JTian L(2020)A Review of Artificial Intelligence for GamesArtificial Intelligence in China10.1007/978-981-15-0187-6_34(298-303)Online publication date: 1-Feb-2020
https://doi.org/10.1007/978-981-15-0187-6_34
Shao KZhu YZhao D(2019)StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer LearningIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2018.28233293:1(73-84)Online publication date: Feb-2019
https://doi.org/10.1109/TETCI.2018.2823329
Smolyakov IBelyaev S(2019)Design of the Software Architecture for Starcraft Video Game on the Basis of Finite State Machines2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus)10.1109/EIConRus.2019.8656866(356-359)Online publication date: Jan-2019
https://doi.org/10.1109/EIConRus.2019.8656866
Tang ZShao KZhu YLi DZhao DHuang T(2018)A Review of Computational Intelligence for StarCraft AI2018 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2018.8628682(1167-1173)Online publication date: Nov-2018
https://doi.org/10.1109/SSCI.2018.8628682
Shao KZhu YZhao D(2017)Cooperative reinforcement learning for multiple units combat in starCraft2017 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2017.8280949(1-6)Online publication date: Nov-2017
https://doi.org/10.1109/SSCI.2017.8280949
Oh ICho HKim K(2017)Playing real-time strategy games by imitating human players' micromanagement skills based on spatial analysisExpert Systems with Applications: An International Journal10.1016/j.eswa.2016.11.02671:C(192-205)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1016/j.eswa.2016.11.026
Robertson GWatson I(2014)A Review of Real‐Time Strategy Game AIAI Magazine10.1609/aimag.v35i4.247835:4(75-104)Online publication date: Dec-2014
https://doi.org/10.1609/aimag.v35i4.2478
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents