Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Multiple Oracle Algorithm to Solve Continuous Games

  • Conference paper
  • First Online:
Decision and Game Theory for Security (GameSec 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13727))

Included in the following conference series:

  • 512 Accesses

Abstract

Continuous games are multiplayer games in which strategy sets are compact and utility functions are continuous. These games typically have a highly complicated structure of Nash equilibria, and numerical methods for the equilibrium computation are known only for particular classes of continuous games, such as two-player polynomial games or games in which pure equilibria are guaranteed to exist. This contribution focuses on the computation and approximation of a mixed strategy equilibrium for the whole class of multiplayer general-sum continuous games. We vastly extend the scope of applicability of the double oracle algorithm, initially designed and proved to converge only for two-player zero-sum games. Specifically, we propose an iterative strategy generation technique, which splits the original problem into the master problem with only a finite subset of strategies being considered, and the subproblem in which an oracle finds the best response of each player. This simple method is guaranteed to recover an approximate equilibrium in finitely many iterations. Further, we argue that the Wasserstein distance (the earth mover’s distance) is the right metric for the space of mixed strategies for our purposes. Our main result is the convergence of this algorithm in the Wasserstein distance to an equilibrium of the original continuous game. The numerical experiments show the performance of our method on several classes of games including randomly generated examples.

The authors acknowledge the support by the project Research Center for Informatics (CZ.02.1.01/0.0/0.0/16_019/0000765).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adam, L., Horčík, R., Kasl, T., Kroupa, T.: Double oracle algorithm for computing equilibria in continuous games. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 5070–5077 (2021)

    Google Scholar 

  2. Başar, T., Olsder, G.: Dynamic Noncooperative Game Theory, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)

    MATH  Google Scholar 

  3. Behnezhad, S., Blum, A., Derakhshan, M., Hajiaghayi, M., Papadimitriou, C.H., Seddighin, S.: Optimal strategies of blotto games: beyond convexity. In: Proceedings of the 2019 ACM Conference on Economics and Computation, EC 2019, ACM, New York, pp. 597–616 (2019)

    Google Scholar 

  4. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Nashua (2016)

    MATH  Google Scholar 

  5. Bošanský, B., Kiekintveld, C., Lisý, V., Pěchouček, M.: An exact double-oracle algorithm for zero-sum extensive-form games with imperfect information. J. Artif. Intell. Res. 51, 829–866 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cai, Y., Candogan, O., Daskalakis, C., Papadimitriou, C.: Zero-sum polymatrix games: a generalization of minmax. Math. Oper. Res. 41(2), 648–655 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chasnov, B., Ratliff, L., Mazumdar, E., Burden, S.: Convergence analysis of gradient-based learning in continuous games. In: Uncertainty in Artificial Intelligence, pp. 935–944. PMLR (2020)

    Google Scholar 

  8. Daskalakis, C., Goldberg, P.W., Papadimitriou, C.H.: The complexity of computing a Nash equilibrium. SIAM J. Comput. 39(1), 195–259 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dunning, I., Huchette, J., Lubin, M.: JuMP: a modeling language for mathematical optimization. SIAM Rev. 59(2), 295–320 (2017). https://doi.org/10.1137/15M1020575

    Article  MathSciNet  MATH  Google Scholar 

  10. Ferris, M.C., Munson, T.S.: Complementarity problems in GAMS and the PATH solver. J. Econ. Dyn. Control 24(2), 165–188 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. Fudenberg, D., Levine, D.: Limit games and limit equilibria. J. Econ. Theor. 38(2), 261–279 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  12. Ganzfried, S.: Algorithm for computing approximate Nash equilibrium in continuous games with application to continuous Blotto. Games 12(2), 47 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gibbs, A.L., Su, F.E.: On choosing and bounding probability metrics. Int. Stat. Rev. 70(3), 419–435 (2002)

    Article  MATH  Google Scholar 

  14. Glicksberg, I.L.: A further generalization of the Kakutani fixed point theorem, with application to Nash equilibrium points. Proc. Am. Math. Soc. 3, 170–174 (1952)

    MathSciNet  MATH  Google Scholar 

  15. Golman, R., Page, S.E.: General blotto: games of allocative strategic mismatch. Public Choice 138(3–4), 279–299 (2009)

    Article  Google Scholar 

  16. Govindan, S., Wilson, R.: A global Newton method to compute Nash equilibria. J. Econ. Theor. 110(1), 65–86 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  17. Govindan, S., Wilson, R.: Computing Nash equilibria by iterated polymatrix approximation. J. Econ. Dyn. Control 28(7), 1229–1241 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE Constraints, vol. 23. Springer, Cham (2008)

    MATH  Google Scholar 

  19. Hofbauer, J., Sorin, S.: Best response dynamics for continuous zero-sum games. Discrete Continuous Dyn. Syst.-Ser. B 6(1), 215 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  20. Kamra, N., Gupta, U., Fang, F., Liu, Y., Tambe, M.: Policy learning for continuous space security games using neural networks. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1103–1112 (2018)

    Google Scholar 

  21. Karlin, S.: Mathematical Methods and Theory in Games, Programming and Economics, Vol. 2: The Theory of Infinite Games. Addison-Wesley Publishing Company, Boston (1959)

    Google Scholar 

  22. Kroupa, T., Vannucci, S., Votroubek, T.: Separable network games with compact strategy sets. In: Bošanský, B., Gonzalez, C., Rass, S., Sinha, A. (eds.) Decision and Game Theory for Security, pp. 37–56. Springer International Publishing, Cham (2021)

    Chapter  Google Scholar 

  23. Laraki, R., Lasserre, J.B.: Semidefinite programming for min-max problems and games. Math. Program. 131(1–2), 305–332 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lasserre, J.B.: An Introduction To Polynomial And Semi-Algebraic Optimization, vol. 52. Cambridge University Press, Cambridge (2015)

    Book  MATH  Google Scholar 

  25. Li, Z., Wellman, M.P.: Evolution strategies for approximate solution of Bayesian games. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 5531–5540 (2021)

    Google Scholar 

  26. McKelvey, R.D., McLennan, A.M., Turocy, T.L.: Gambit: Software tools for game theory. Version 16.0.1 (2016)

    Google Scholar 

  27. McMahan, H.B., Gordon, G.J., Blum, A.: Planning in the presence of cost functions controlled by an adversary. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 536–543 (2003)

    Google Scholar 

  28. Mertikopoulos, P., Lecouat, B., Zenati, H., Foo, C.S., Chandrasekhar, V., Piliouras, G.: Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile (2018)

    Google Scholar 

  29. Mertikopoulos, P., Zhou, Z.: Learning in games with continuous action sets and unknown payoff functions. Math. Program. 173(1), 465–507 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  30. Niu, L., Sahabandu, D., Clark, A., Poovendran, R.: A game-theoretic framework for controlled islanding in the presence of adversaries. In: Bošanský, B., Gonzalez, C., Rass, S., Sinha, A. (eds.) GameSec 2021. LNCS, vol. 13061, pp. 231–250. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90370-1_13

    Chapter  Google Scholar 

  31. Panaretos, V.M., Zemel, Y.: An Invitation To Statistics In Wasserstein Space. Springer Nature, Cham (2020)

    Book  MATH  Google Scholar 

  32. Parrilo, P.: Polynomial games and sum of squares optimization. In: 2006 45th IEEE Conference on Decision and Control, pp. 2855–2860 (2006)

    Google Scholar 

  33. Peyré, G., Cuturi, M.: Computational optimal transport: with applications to data science. Found. Trends Mach. Learn. 11(5–6), 355–607 (2019)

    Article  MATH  Google Scholar 

  34. Rehbeck, J.: Note on unique Nash equilibrium in continuous games. Games Econ. Behav. 110, 216–225 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  35. Roussillon, B., Loiseau, P.: Scalable optimal classifiers for adversarial settings under uncertainty. In: Bošanský, B., Gonzalez, C., Rass, S., Sinha, A. (eds.) GameSec 2021. LNCS, vol. 13061, pp. 80–97. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90370-1_5

    Chapter  Google Scholar 

  36. Stein, N.D., Ozdaglar, A., Parrilo, P.A.: Separable and low-rank continuous games. Int. J. Game Theor. 37(4), 475–504 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006). https://doi.org/10.1007/s10107-004-0559-y. https://doi.org/10.1007/s10107-004-0559-y

  38. Weisser, T., Legat, B., Coey, C., Kapelevich, L., Vielma, J.P.: Polynomial and moment optimization in Julia and JuMP. In: JuliaCon (2019). https://pretalx.com/juliacon2019/talk/QZBKAU/

  39. Xu, L., Perrault, A., Fang, F., Chen, H., Tambe, M.: Robust reinforcement learning under minimax regret for green security. In: Uncertainty in Artificial Intelligence, pp. 257–267. PMLR (2021)

    Google Scholar 

  40. Yasodharan, S., Loiseau, P.: Nonzero-sum adversarial hypothesis testing games. In: Advances in Neural Information Processing Systems, pp. 7310–7320 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Artificial Intelligence Center, Department of Computer Science, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic

    Tomáš Kroupa & Tomáš Votroubek

Authors
  1. Tomáš Kroupa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomáš Votroubek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomáš Kroupa .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Fei Fang

  2. University of Chicago, Chicago, IL, USA

    Haifeng Xu

  3. Université d'Avignon, Avignon, France

    Yezekael Hayel

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kroupa, T., Votroubek, T. (2023). Multiple Oracle Algorithm to Solve Continuous Games. In: Fang, F., Xu, H., Hayel, Y. (eds) Decision and Game Theory for Security. GameSec 2022. Lecture Notes in Computer Science, vol 13727. Springer, Cham. https://doi.org/10.1007/978-3-031-26369-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26369-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26368-2

  • Online ISBN: 978-3-031-26369-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation