research-article

Bounds and dynamics for empirical game theoretic analysis

Authors:

Julien Perolat,

Richard Everett,

Csaba Szepesvári,

Thore GraepelAuthors Info & Claims

Autonomous Agents and Multi-Agent Systems, Volume 34, Issue 1

https://doi.org/10.1007/s10458-019-09432-y

Published: 04 December 2019 Publication History

Abstract

This paper provides several theoretical results for empirical game theory. Specifically, we introduce bounds for empirical game theoretical analysis of complex multi-agent interactions. In doing so we provide insights in the empirical meta game showing that a Nash equilibrium of the estimated meta-game is an approximate Nash equilibrium of the true underlying meta-game. We investigate and show how many data samples are required to obtain a close enough approximation of the underlying game. Additionally, we extend the evolutionary dynamics analysis of meta-games using heuristic payoff tables (HPTs) to asymmetric games. The state-of-the-art has only considered evolutionary dynamics of symmetric HPTs in which agents have access to the same strategy sets and the payoff structure is symmetric, implying that agents are interchangeable. Finally, we carry out an empirical illustration of the generalised method in several domains, illustrating the theory and evolutionary dynamics of several versions of the AlphaGo algorithm (symmetric), the dynamics of the Colonel Blotto game played by human players on Facebook (symmetric), the dynamics of several teams of players in the capture the flag game (symmetric), and an example of a meta-game in Leduc Poker (asymmetric), generated by the policy-space response oracle multi-agent learning algorithm.

References

[1]

Avis D, Rosenberg G, Savani R, and von Stengel BEnumeration of nash equilibria for two-player gamesEconomic Theory2010429-372551727

[2]

Bloembergen D, Hennes D, McBurney P, and Tuyls K Trading in markets with noisy information: An evolutionary analysis Connection Science 2015 27 253-268

[3]

Bloembergen D, Tuyls K, Hennes D, and Kaisers MEvolutionary dynamics of multi-agent learning: A surveyJournal of Artificial Intelligence Research (JAIR)201553659-6973389566

[4]

Borel, E. (1921). La théorie du jeu les équations intégrales à noyau symétrique. comptes rendus de l’académie, 173, 1304–1308. english translation by savage, l.: The theory of play and integral equations with skew symmetric kernels. Econometrica 21, 97–100 (1953).

[5]

Brinkman, E., & Wellman, M. (2016). Shading and efficiency in limit-order markets. In Proceedings of the IJCAI-16 workshop on algorithmic game theory.

[6]

Cassell BA and Wellman M Giardini F and Amblard F EGTAOnline: An experiment manager for simulation-based game studies Multi-Agent-Based Simulation XIII, Lecture Notes in Computer Science 2013 Berlin Springer

[7]

Coulom, R. (2008). Whole-history rating: A Bayesian rating system for players of time-varying strength. In Computers and games, 6th international conference, CG 2008, Beijing, China, September 29–October 1, 2008. Proceedings (pp. 113–124).

[8]

Elo AE The rating of chess players, past and present 1978 Bronx Ishi Press International

[9]

Erev I and Roth AEMulti-agent learning and the descriptive value of simple modelsArtificial Intelligence20071717423-4282332291

[10]

Gintis H Game theory evolving 2009 2 Princeton, NJ University Press

[11]

Hennes, D., Claes, D., & Tuyls, K. (2013). Evolutionary advantage of reciprocity in collision avoidance. In Proceedings of the AAMAS 2013 workshop on autonomous robots and multirobot systems (ARMS 2013)

[12]

Hofbauer J and Sigmund K Evolutionary games and population dynamics 1998 Cambridge Cambridge University Press

[13]

Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castaneda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2018). Human-level performance in first-person multiplayer games with population-based deep reinforcement learning. arXiv preprint arXiv:1807.01281.

[14]

Jecmen, S., Brinkman, E., & Sinha, A. (2018). Bounding regret in simulated games. In: ICML workshop on exploration in RL.

[15]

Julian Schvartzman L and Wellman MP Stronger CDA strategies through empirical game-theoretic analysis and reinforcement learning AAMAS 2009 1 249-256

[16]

Kaisers, M., Tuyls, K., Thuijsman, F., & Parsons, S. (2008). Auction analysis by normal form game approximation. In Proceedings of the 2008 IEEE/WIC/ACM international conference on intelligent agent technology, Sydney, NSW, Australia, December 9–12, 2008 (pp. 447–450).

[17]

Kohli, P., Kearns, M., Bachrach, Y., Herbrich, R., Stillwell, D., & Graepel, T. (2012). Colonel blotto on facebook: the effect of social relations on strategic interaction. In Web science 2012, WebSci ’12, Evanston, IL, USA, June 22–24, 2012 (pp. 141–150).

[18]

Lanctot M, Zambaldi V, Gruslys A, Lazaridou A, Tuyls K, Perolat J, Silver D, and Graepel T Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, and Garnett R A unified game-theoretic approach to multiagent reinforcement learning Advances in neural information processing systems 30 2017 Berlin Springer 4190-4203

[19]

Maynard Smith J and Price GR The logic of animal conflicts Nature 1973 246 15-18

[20]

Nguyen, T., Wright, M., Wellman, M., & Singh, S. (2017). Multi-stage attack graph security games: Heuristic strategies, with empirical game-theoretic analysis. In Proceedings of the fourth ACM workshop on moving target defense.

[21]

Nijssen P and Winands MH Monte carlo tree search for the hide-and-seek game scotland yard IEEE Transactions on Computational Intelligence and AI in Games 2012 4 4 282-294

[22]

Phelps, S., Cai, K., McBurney, P., Niu, J., Parsons, S., & Sklar, E. (2007). Auctions, evolution, and multi-agent learning. In Adaptive agents and multi-agent systems III. Adaptation and multi-agent learning, 5th, 6th, and 7th European symposium, ALAMAS 2005-2007 on adaptive and learning agents and multi-agent systems, Revised Selected Papers (pp. 188–210).

[23]

Phelps, S., Parsons, S., & McBurney, P. (2004). An evolutionary game-theoretic comparison of two double-auction market designs. In Agent-mediated electronic commerce VI, theories for and engineering of distributed mechanisms and systems, AAMAS 2004 Workshop, AMEC 2004, New York, NY, USA, July 19, 2004, Revised Selected Papers (pp. 101–114).

[24]

Ponsen M, Tuyls K, Kaisers M, and Ramon J An evolutionary game-theoretic analysis of poker strategies Entertainment Computing 2009 1 1 39-45

[25]

Prakash, A., & Wellman, M. (2015). Empirical game-theoretic analysis for moving target defense. In Proceedings of the second ACM workshop on moving target defense

[26]

Rosenfeld A and Kraus S Predicting human decision-making: From prediction to action. Synthesis lectures on artificial intelligence and machine learning 2018 San Rafael Morgan & Claypool Publishers

[27]

Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap TP, Leach M, Kavukcuoglu K, Graepel T, and Hassabis D Mastering the game of go with deep neural networks and tree search Nature 2016 529 7587 484-489

[28]

Southey, F., Bowling, M., Larson, B., Piccione, C., Burch, N., Billings, D., & Rayner, C. (2005). Bayes’ bluff: Opponent modelling in poker. In Proceedings of the twenty-first conference on uncertainty in artificial intelligence (UAI-05).

[29]

Stone P and Veloso M Towards collaborative and adversarial learning: A case study in robotic soccer International Journal of Human-Computer Studies 1998 48 1 83-104

[30]

Tuyls K and Parsons SWhat evolutionary game theory tells us about multiagent learningArtificial Intelligence20071717406-4162332289

[31]

Tuyls, K., Pérolat, J., Lanctot, M., Leibo, J. Z., & Graepel, T. (2018). A generalised method for empirical game theoretic analysis. In International foundation for autonomous agents and multiagent systems (AAMAS), Richland, SC, USA/ACM (pp. 77–85).

[32]

Tuyls K, Perolat J, Lanctot M, Savani R, Leibo J, Ord T, Graepel T, and Legg S Symmetric decomposition of asymmetric games Scientific Reports 2018 8 1 1015

[33]

Tuyls, K., Verbeeck, K., & Lenaerts, T. (2003). A selection-mutation model for q-learning in multi-agent systems. In The second international joint conference on autonomous agents & multiagent systems, AAMAS 2003, July 14–18, 2003, Melbourne, Victoria, Australia, Proceedings (pp. 693–700).

[34]

Vorobeychik YProbabilistic analysis of simulation-based gamesACM Transactions on Modeling and Computer Simulation2010203161386.91023

[35]

Vorobeychik, Y., & Wellman, M. P. (2008). Stochastic search methods for nash equilibrium approximation in simulation-based games. In Proceedings of the seventh international conference on autonomous agents and multiagent systems (AAMAS) (pp. 1055–1062).

[36]

Vorobeychik Y, Wellman MP, and Singh S Learning payoff functions in infinite games Machine Learning 2007 67 145-168

[37]

Wah, E., Hurd, D., & Wellman, M. (2015). Strategic market choice: Frequent call markets vs. continuous double auctions for fast and slow traders. In Proceedings of the third EAI conference on auctions, market mechanisms, and their applications.

[38]

Wah E, Wright M, and Wellman MWelfare effects of market making in continuous double auctionsJournal of Artificial Intelligence Research201759613-6503687596

[39]

Walsh, W. E., Das, R., Tesauro, G., & Kephart, J. (2002). Analyzing complex strategic interactions in multi-agent games. In AAAI-02 workshop on game theoretic and decision theoretic agents, 2002.

[40]

Walsh, W. E., Parkes, D. C., & Das, R. (2003). Choosing samples to compute heuristic-strategy nash equilibrium. In Proceedings of the fifth workshop on agent-mediated electronic commerce.

[41]

Wang, X., Vorobeychik, Y., & Wellman, M. (2018). A cloaking mechanism to mitigate market manipulation. In Proceedings of the 27th international joint conference on artificial intelligence (pp. 541–547).

[42]

Weibull J Evolutionary game theory 1997 Cambridge MIT Press

[43]

Wellman, M., Kim, T., & Duong, Q. (2013). Analyzing incentives for protocol compliance in complex domains: A case study of introduction-based routing. In Proceedings of the 12th workshop on the economics of information security.

[44]

Wellman, M. P. (2006). Methods for empirical game-theoretic analysis. In Proceedings, The twenty-first national conference on artificial intelligence and the eighteenth innovative applications of artificial intelligence conference July 16–20, 2006, Boston, Massachusetts, USA (pp. 1552–1556).

[45]

Wiedenbeck, B., Cassell, B. A., & Wellman, M. P. (2014). Bootstrap statistics for empirical games. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems (pp. 597–604). International Foundation for Autonomous Agents and Multiagent Systems.

[46]

Wiedenbeck, B., & Wellman, M. (2012). Scaling simulation-based game analysis through deviation-preserving reduction. In Proceedings of the eleventh international conference on autonomous agents and multiagent systems (AAMAS).

[47]

Wright, M. (2016). Using reinforcement learning to validate empirical game-theoretic analysis: A continuous double auction study. CoRR arXiv:1604.06710.

[48]

Wright, M., Venkatesan, S., Albenese, M., & Wellman, M. (2016). Moving target defense against DDoS attacks: An empirical game-theoretic analysis. In Proceedings of the third ACM workshop on moving target defense.

[49]

Wright, M., & Wellman, M. P. (2018). Evaluating the stability of non-adaptive trading in continuous double auctions. AAMAS, 614–622.

[50]

Zeeman, E. C., (1980). Population dynamics from game theory. In Global theory of dynamical systems, Springer, Berlin, Heidelberg (pp. 471–497).

[51]

Zeeman EDynamics of the evolution of animal conflictsTheoretical Biology198189249-270630636

Cited By

Wang YWellman MDastani MSichman JAlechina NDignum V(2024)Generalized Response Objectives for Strategy Exploration in Empirical Game-Theoretic AnalysisProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663052(1892-1900)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663052
Wu FThiery TLeonardos SVentre C(2024)To Compete or Collude: Bidding Incentives in Ethereum Block Building AuctionsProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698629(813-821)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698629
Wang YWellman MAgmon NAn BRicci AYeoh W(2023)Regularization for Strategy Exploration in Empirical Game-Theoretic AnalysisProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems10.5555/3545946.3598975(2484-2486)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.5555/3545946.3598975
Show More Cited By

Index Terms

Bounds and dynamics for empirical game theoretic analysis

Index terms have been assigned to the content through auto-classification.

Recommendations

A Generalised Method for Empirical Game Theoretic Analysis
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems

This paper provides theoretical bounds for empirical game theoretical analysis of complex multi-agent interactions. We provide insights in the empirical meta game showing that a Nash equilibrium of the meta-game is an approximate Nash equilibrium of the ...
Two population three-player prisoner's dilemma game

We present a two population three-player prisoner's dilemma.Finite state automata are used.Different types of Tit for Tat strategies.All possible payoffs are computed. Due to the computational advantage in symmetric games, most researches have focused ...
Replicator dynamics of an N-player snowdrift game with delayed payoffs
Highlights
- The replicator dynamics for an N-player snowdrift game are examined under the context of a social-type time delay.
Abstract
We consider a version of the N-player snowdrift game in which the payoffs obtained by the players are delayed. The delay in payoffs is shown to lead to a Hopf bifurcation with an associated critical value of the time delay in the ...

Comments

Information & Contributors

Information

Published In

cover image Autonomous Agents and Multi-Agent Systems

Autonomous Agents and Multi-Agent Systems Volume 34, Issue 1

Mar 2020

1000 pages

ISSN:1387-2532

Issue’s Table of Contents

© The Author(s) 2019.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 04 December 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YWellman MDastani MSichman JAlechina NDignum V(2024)Generalized Response Objectives for Strategy Exploration in Empirical Game-Theoretic AnalysisProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663052(1892-1900)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663052
Wu FThiery TLeonardos SVentre C(2024)To Compete or Collude: Bidding Incentives in Ethereum Block Building AuctionsProceedings of the 5th ACM International Conference on AI in Finance10.1145/3677052.3698629(813-821)Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1145/3677052.3698629
Wang YWellman MAgmon NAn BRicci AYeoh W(2023)Regularization for Strategy Exploration in Empirical Game-Theoretic AnalysisProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems10.5555/3545946.3598975(2484-2486)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.5555/3545946.3598975
Wang YWellman MAgmon NAn BRicci AYeoh W(2023)Empirical Game-Theoretic Analysis for Mean Field GamesProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems10.5555/3545946.3598742(1025-1033)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.5555/3545946.3598742
Cousins CMishra BAreyan Viqueira EGreenwald AAgmon NAn BRicci AYeoh W(2023)Learning Properties in Simulation-Based GamesProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems10.5555/3545946.3598647(272-280)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.5555/3545946.3598647
Zhao LPolukarov MVentre C(2023)Liquidity and Solvency Risks in Financial NetworksProceedings of the Fourth ACM International Conference on AI in Finance10.1145/3604237.3626840(210-218)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3604237.3626840
Wen YChen HYang YLi MTian ZChen XWang J(2022)A Game-Theoretic Approach to Multi-agent Trust Region OptimizationDistributed Artificial Intelligence10.1007/978-3-031-25549-6_6(74-87)Online publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1007/978-3-031-25549-6_6
Czarnecki WGidel GTracey BTuyls KOmidshafiei SBalduzzi DJaderberg MLarochelle HRanzato MHadsell RBalcan MLin H(2020)Real world games look like spinning topsProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3497187(17443-17454)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3497187

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents