extended-abstract

Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity

Authors:

Marta Garnelo,

Wojciech Marian Czarnecki,

David BalduzziAuthors Info & Claims

AAMAS '21: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems

Pages 1501 - 1503

Published: 03 May 2021 Publication History

Get Access

Abstract

Strategic diversity is often essential in games: in multi-player games, for example, evaluating a player against a diverse set of strategies will yield a more accurate estimate of its performance. Furthermore, in games with non-transitivities diversity allows a player to cover several winning strategies. However, despite the significance of strategic diversity, training agents that exhibit diverse behaviour remains a challenge. In this paper we study how to construct diverse populations of agents by carefully structuring how individuals within a population interact. Our approach is based on interaction graphs, which control the flow of information between agents during training and can encourage agents to specialise on different strategies, leading to improved overall performance. We provide evidence for the importance of diversity in multi-agent training and analyse the effect of applying different interaction graphs on the training trajectories, diversity and performance of populations in a range of games.

References

[1]

David Balduzzi, Marta Garnelo, Yoram Bachrach, Wojciech M Czarnecki, Julien Perolat, Max Jaderberg, and Thore Graepel. 2019. Open-ended learning in symmetric zero-sum games. International Conference on Machine Learning (2019).

Google Scholar

[2]

Marta Garnelo, Wojciech Marian Czarnecki, Siqi Liu, Dhruva Tirumala, Junhyuk Oh, Gauthier Gidel, William Hawkins, Hado van Hasselt, and David Balduzzi. 2021. Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity. arXiv preprint (2021).

Google Scholar

[3]

Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, and Thore Graepel. 2017. A unified game theoretic approach to multiagent reinforcement learning. In Advances in Neural Information Processing Systems. 4190--4203.

Google Scholar

[4]

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017. Mastering the game of go without human knowledge. Nature 550, 7676 (2017), 354--359.

Google Scholar

[5]

Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575, 7782 (2019), 350--354.

Google Scholar

[6]

Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, et al. 2017. Starcraft ii: A new challenge for reinforcement learning. arXiv preprint arXiv:1708.04782 (2017).

Google Scholar

Cited By

View all

Lorraine JVicol PParker-Holder JKachman TMetz LFoerster JPelachaud CTaylor MFaliszewski PMascardi V(2022)Lyapunov Exponents for Diversity in Differentiable GamesProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3535945(842-852)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.5555/3535850.3535945

Index Terms

Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
        Multi-agent reinforcement learning

Recommendations

Maintaining Healthy Population Diversity Using Adaptive Crossover, Mutation, and Selection

This paper presents ACROMUSE, a novel genetic algorithm (GA) which adapts crossover, mutation, and selection parameters. ACROMUSEs objective is to create and maintain a diverse population of highly-fit (healthy) individuals, capable of adapting quickly ...
Analysis of diversity mechanisms for optimisation in dynamic environments with low frequencies of change

Evolutionary dynamic optimisation has become one of the most active research areas in evolutionary computation. We consider the Balance function for which the poor expected performance of the ( 1 + 1 ) EA at low frequencies of change has been shown in ...
Population Diversity as a Selection Factor: Improving Fitness by Increasing Diversity
GECCO '16 Companion: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion

Evolutionary algorithms search for problem solutions by selecting individuals for survival and reproduction with a bias towards higher fitness. Such biases may lead to premature convergence on sub-optimal solutions. A bias toward greater diversity can ...

Comments

Information & Contributors

Information

Published In

AAMAS '21: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems

May 2021

1899 pages

ISBN:9781450383073

General Chairs:
Frank Dignum
Umeå University, Sweden
,
Alessio Lomuscio
Imperial College London, UK
,
Program Chairs:
Ulle Endriss
University of Amsterdam, Netherlands
,
Ann Nowé
Vrije Universiteit Brussel, Belgium

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 03 May 2021

Check for updates

Author Tags

Qualifiers

Extended-abstract

Conference

AAMAS '21

Sponsor:

SIGAI

AAMAS '21: 20th International Conference on Autonomous Agents and Multiagent Systems

May 3 - 7, 2021

Virtual Event, United Kingdom

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
43
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lorraine JVicol PParker-Holder JKachman TMetz LFoerster JPelachaud CTaylor MFaliszewski PMascardi V(2022)Lyapunov Exponents for Diversity in Differentiable GamesProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3535945(842-852)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.5555/3535850.3535945

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Maintaining Healthy Population Diversity Using Adaptive Crossover, Mutation, and Selection

Analysis of diversity mechanisms for optimisation in dynamic environments with low frequencies of change

Population Diversity as a Selection Factor: Improving Fitness by Increasing Diversity

Comments

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Maintaining Healthy Population Diversity Using Adaptive Crossover, Mutation, and Selection

Analysis of diversity mechanisms for optimisation in dynamic environments with low frequencies of change

Population Diversity as a Selection Factor: Improving Fitness by Increasing Diversity

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations