poster

Public Access

A hybrid search agent in pommerman

Authors:

Julian TogeliusAuthors Info & Claims

FDG '18: Proceedings of the 13th International Conference on the Foundations of Digital Games

Article No.: 46, Pages 1 - 4

https://doi.org/10.1145/3235765.3235812

Published: 07 August 2018 Publication History

PDF eReader

Abstract

In this paper, we explore the possibility of search-based agents in games with resource-intensive forward models. We implemented a player agent in the Pommerman framework and put it against the baseline agent to measure its performance. We implemented a heuristic agent and improved it by enabling depth-limited tree search in specific gameplay moments. We also compared different node selection methods during depth-limited tree search. Our result shows that depth-limited tree search is still viable when presented with inefficient forward models and exploitation-driven selection method is the most efficient in this specific domain.

References

[1]

Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games 4, 1 (2012), 1--43.

Crossref

Google Scholar

[2]

Christopher Clark and Amos Storkey. 2015. Training deep convolutional neural networks to play go. In International Conference on Machine Learning. 1766--1774.

Digital Library

Google Scholar

[3]

Markus Enzenberger, Martin Muller, Broderick Arneson, and Richard Segal. 2010. Fuego - an open-source framework for board games and Go engine based on Monte Carlo tree search. IEEE Transactions on Computational Intelligence and AI in Games 2, 4 (2010), 259--270.

Crossref

Google Scholar

[4]

Hilmar Finnsson and Yngvi Björnsson. 2011. Game-tree properties and MCTS performance. In IJCAI, Vol. 11. 23--30.

Google Scholar

[5]

Edmond S L Ho and Taku Komura. 2010. A finite state machine based on topology coordinates for wrestling games. Computer Animation and Virtual Worlds 22, 5 (2010), 435--443.

Digital Library

Google Scholar

[6]

Shih-Chieh Huang and Martin Müller. 2013. Investigating the limits of Monte-Carlo tree search methods in computer Go. In International Conference on Computers and Games. Springer, 39--48.

Google Scholar

[7]

Emil Juul Jacobsen, Rasmus Greve, and Julian Togelius. 2014. Monte mario: platforming with mcts. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation. ACM, 293--300.

Digital Library

Google Scholar

[8]

Tom Pepels, Tristan Cazenave, and Mark HM Winands. 2015. Sequential halving for partially observable games. In Computer Games. Springer, 16--29.

Google Scholar

[9]

Christian P Robert. 2004. Monte carlo methods. Wiley Online Library.

Google Scholar

[10]

John E Savage. 1998. Models of computation. Vol. 136. Addison-Wesley Reading, MA.

Google Scholar

[11]

Dennis Soemers. 2014. Tactical planning using MCTS in the game of StarCraft. Ph.D. Dissertation. MasterâĂ&Zacute;s thesis, Department of Knowledge Engineering, Maastricht University.

Google Scholar

[12]

Mark Oude Veldhuis. 2010. Artificial Intelligence techniques used in First-Person Shooter and Real-Time Strategy games. In Human Media Seminar: Designing Entertainment Interaction, Vol. 2011. Citeseer.

Google Scholar

[13]

Vincent Vidal and others. 2004. A Lookahead Strategy for Heuristic Search Planning. In ICAPS. 150--160.

Digital Library

Google Scholar

[14]

Shuyi Zhang and Michael Buro. 2017. Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events. In Computational Intelligence and Games (CIG), 2017 IEEE Conference on. IEEE, 309--316.

Crossref

Google Scholar

Cited By

View all

Charity MEarle SRajesh DWilson MTogelius J(2024)Amorphous Fortress: Exploring Emergent Behavior and Complexity in Multi-Agent 0-Player Games2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10612153(01-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10612153
Shelke OMeisheri HKhadilkar H(2022)Identifying efficient curricula for reinforcement learning in complex environments with a fixed computational budgetProceedings of the 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)10.1145/3493700.3493709(81-89)Online publication date: 8-Jan-2022
https://dl.acm.org/doi/10.1145/3493700.3493709
Świechowski MGodlewski KSawicki BMańdziuk J(2022)Monte Carlo Tree Search: a review of recent modifications and applicationsArtificial Intelligence Review10.1007/s10462-022-10228-y56:3(2497-2562)Online publication date: 19-Jul-2022
https://doi.org/10.1007/s10462-022-10228-y
Show More Cited By

Index Terms

A hybrid search agent in pommerman
1. Applied computing
  1. Computers in other domains
    1. Personal computers and PC applications
      1. Computer games
2. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Game tree search

Recommendations

A unified framework for partial and hybrid search methods in constraint programming

We present a library called ToOLS for the design of complex tree search algorithms in constraint programming (CP). We separate the description of a search algorithm into three parts: a refinement-based search scheme that defines a complete search tree, a ...
Iterative-deepening search with on-line tree size prediction

The memory requirements of best-first graph search algorithms such as A* often prevent them from solving large problems. The best-known approach for coping with this issue is iterative deepening, which performs a series of bounded depth-first searches. ...
Tree Search in Major/Minor Loop Magnetic Bubble Memories

In this paper we analyze various search schemes in a major/minor loop bubble memory. Specifically, we study balanced tree search, one-sided height-balanced tree search, and one-sided K-Keight-balanced tree search. Two parameters are of interest in the ...

Comments

Information & Contributors

Information

Published In

FDG '18: Proceedings of the 13th International Conference on the Foundations of Digital Games

August 2018

503 pages

ISBN:9781450365710

DOI:10.1145/3235765

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2018

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

Conference

FDG '18

FDG '18: Foundations of Digital Games 2018

August 7 - 10, 2018

Malmö, Sweden

Acceptance Rates

FDG '18 Paper Acceptance Rate 39 of 95 submissions, 41%;

Overall Acceptance Rate 152 of 415 submissions, 37%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
591
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)9

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Charity MEarle SRajesh DWilson MTogelius J(2024)Amorphous Fortress: Exploring Emergent Behavior and Complexity in Multi-Agent 0-Player Games2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10612153(01-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10612153
Shelke OMeisheri HKhadilkar H(2022)Identifying efficient curricula for reinforcement learning in complex environments with a fixed computational budgetProceedings of the 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)10.1145/3493700.3493709(81-89)Online publication date: 8-Jan-2022
https://dl.acm.org/doi/10.1145/3493700.3493709
Świechowski MGodlewski KSawicki BMańdziuk J(2022)Monte Carlo Tree Search: a review of recent modifications and applicationsArtificial Intelligence Review10.1007/s10462-022-10228-y56:3(2497-2562)Online publication date: 19-Jul-2022
https://doi.org/10.1007/s10462-022-10228-y
Swiechowski MLewinski DTyl R(2021)Combining Utility AI and MCTS Towards Creating Intelligent Agents in Video Games, with the Use Case of Tactical Troops: Anthracite Shift2021 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI50451.2021.9660170(1-8)Online publication date: 5-Dec-2021
https://doi.org/10.1109/SSCI50451.2021.9660170
Yang HLi SXu XLiu XMeng ZZhang Y(2021)Efficient Searching With MCTS and Imitation Learning: A Case Study in PommermanIEEE Access10.1109/ACCESS.2021.30613139(48851-48859)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3061313
Barde PRoy JJeon WPineau JNowrouzezahrai DPal CLarochelle HRanzato MHadsell RBalcan MLin H(2020)Adversarial soft advantage fittingProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496758(12334-12344)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496758
Agarwal SWallner GBeck F(2020)Bombalytics: Visualization of Competition and Collaboration Strategies of Players in a Bomb Laying GameComputer Graphics Forum10.1111/cgf.1396539:3(89-100)Online publication date: 18-Jul-2020
https://doi.org/10.1111/cgf.13965
Resnick CGao CMárton GOsogami TPang LTakahashi T(2019)Pommerman & NeurIPS 2018The NeurIPS '18 Competition10.1007/978-3-030-29135-8_2(11-36)Online publication date: 30-Nov-2019
https://doi.org/10.1007/978-3-030-29135-8_2

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

A unified framework for partial and hybrid search methods in constraint programming

Iterative-deepening search with on-line tree size prediction

Tree Search in Major/Minor Loop Magnetic Bubble Memories

Comments

Information

Published In

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations