Article

RV_σ(t): a unifying approach to performance and convergence in online multiagent learning

Authors:

Bikramjit Banerjee and

Jing PengAuthors Info & Claims

AAMAS '06: Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems

May 2006

Pages 798 - 800

https://doi.org/10.1145/1160633.1160775

Published: 08 May 2006 Publication History

Get Access

Abstract

We present a new multiagent learning algorithm (RV_σ(t) that can guarantee both no-regret performance (all games) and policy convergence (some games of arbitrary size). Unlike its predecessor ReDVaLeR, it (1) does not need to distinguish whether its opponents are self-play or otherwise non-stationary, (2) is allowed to know its portion of any equilibrium that, we argue, leads to convergence in some games in addition to no-regret. Although the regret of RV_σ(t) is analyzed in continuous time, we show that it grows slower than in other no-regret techniques like GIGA and GIGA-WoLF. We show that RV_σ(t) can converge to coordinated behavior in coordination games, while GIGA, GIGA-WoLF may converge to poorly coordinated (mixed) behaviors.

References

[1]

B. Banerjee and J. Peng. Performance bounded reinforcement learning in strategic intercations. In Proceedings of the 19th National Conference on Artificial Intelligence (AAAI-04), pages 2--7, San Jose, CA, 2004. AAAI Press.

Digital Library

Google Scholar

[2]

B. Banerjee and J. Peng. Convergence of no-regret learning in multiagent systems. In Proceedings of the First International Workshop on Learning and Adaptation in Multiagent Systems (LAMAS), Utrecht, The Netherlands, 2005. Held in conjunction with AAMAS-05.

Google Scholar

[3]

M. Bowling. Convergence and no-regret in multiagent learning. In Proceedings of NIPS 2004/5, 2005.

Google Scholar

[4]

M. Bowling and M. Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136:215--250, 2002.

Digital Library

Google Scholar

[5]

V. Conitzer and T. Sandholm. AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. In Proceedings of the 20th International Conference on Machine Learning, 2003.

Google Scholar

[6]

M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning, Washington DC, 2003.

Google Scholar

Cited By

View all

Conitzer V(2009)Approximation guarantees for fictitious play2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton)10.1109/ALLERTON.2009.5394918(636-643)Online publication date: Sep-2009
https://doi.org/10.1109/ALLERTON.2009.5394918
Chakraborty DStone P(2008)Online multiagent learning against memory bounded adversariesProceedings of the 2008th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I10.5555/3120828.3120864(211-226)Online publication date: 15-Sep-2008
https://dl.acm.org/doi/10.5555/3120828.3120864
Hines GLarson K(2008)Using adaptive consultation of experts to improve convergence rates in multiagent learningProceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 310.5555/1402821.1402866(1337-1340)Online publication date: 12-May-2008
https://dl.acm.org/doi/10.5555/1402821.1402866
Show More Cited By

Recommendations

Reaching pareto-optimality in prisoner's dilemma using conditional joint action learning

We consider the learning problem faced by two self-interested agents repeatedly playing a general-sum stage game. We assume that the players can observe each other's actions but not the payoffs received by the other player. The concept of Nash ...
Read More
Game of Thrones: Fully Distributed Learning for Multiplayer Bandits
We consider an N-player multiarmed bandit game in which each player chooses one out of M arms for T turns. Each player has different expected rewards for the arms, and the instantaneous rewards are independent and identically distributed or Markovian. ...
Read More
Game theory-based opponent modeling in large imperfect-information games
AAMAS '11: The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2

We develop an algorithm for opponent modeling in large extensive-form games of imperfect information. It works by observing the opponent's action frequencies and building an opponent model by combining information from a precomputed equilibrium strategy ...
Read More

Comments

Information & Contributors

Information

Published In

AAMAS '06: Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems

May 2006

1631 pages

ISBN:1595933034

DOI:10.1145/1160633

General Chairs:
Hideyuki Nakashima
Future University - Hakodate, Japan
,
Michael Wellman
University of Michigan
,
Program Chairs:
Gerhard Weiss
Technical University Munich, Germany
,
Peter Stone
The University of Texas at Austin

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

AAMAS06

Sponsor:

IFMAS
SIGAI
ATAL

AAMAS06: AAMAS '06 - 5th International Joint Conference on Autonomous Agents and Multi-agent Systems 2006

May 8 - 12, 2006

Japan, Hakodate

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
193
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

View all

Conitzer V(2009)Approximation guarantees for fictitious play2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton)10.1109/ALLERTON.2009.5394918(636-643)Online publication date: Sep-2009
https://doi.org/10.1109/ALLERTON.2009.5394918
Chakraborty DStone P(2008)Online multiagent learning against memory bounded adversariesProceedings of the 2008th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I10.5555/3120828.3120864(211-226)Online publication date: 15-Sep-2008
https://dl.acm.org/doi/10.5555/3120828.3120864
Hines GLarson K(2008)Using adaptive consultation of experts to improve convergence rates in multiagent learningProceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 310.5555/1402821.1402866(1337-1340)Online publication date: 12-May-2008
https://dl.acm.org/doi/10.5555/1402821.1402866
Chakraborty DSen SPadgham LParkes D(2008)MB-AIM-FSIProceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 110.5555/1402383.1402438(371-378)Online publication date: 12-May-2008
https://dl.acm.org/doi/10.5555/1402383.1402438
Chakraborty DStone P(2008)Online Multiagent Learning against Memory Bounded AdversariesMachine Learning and Knowledge Discovery in Databases10.1007/978-3-540-87479-9_32(211-226)Online publication date: 2008
https://doi.org/10.1007/978-3-540-87479-9_32

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Reaching pareto-optimality in prisoner's dilemma using conditional joint action learning

Game of Thrones: Fully Distributed Learning for Multiplayer Bandits

Game theory-based opponent modeling in large imperfect-information games