Article

Cooperating with unknown teammates in complex domains: a robot soccer case study of ad hoc teamwork

Authors:

Samuel Barrett,

Peter StoneAuthors Info & Claims

AAAI'15: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence

Pages 2010 - 2016

Published: 25 January 2015 Publication History

Abstract

Many scenarios require that robots work together as a team in order to effectively accomplish their tasks. However, pre-coordinating these teams may not always be possible given the growing number of companies and research labs creating these robots. Therefore, it is desirable for robots to be able to reason about ad hoc teamwork and adapt to new teammates on the fly. Past research on ad hoc teamwork has focused on relatively simple domains, but this paper demonstrates that agents can reason about ad hoc teamwork in complex scenarios. To handle these complex scenarios, we introduce a new algorithm, PLASTIC-Policy, that builds on an existing ad hoc teamwork approach. Specifically, PLASTIC- Policy learns policies to cooperate with past teammates and reuses these policies to quickly adapt to new teammates. This approach is tested in the 2D simulation soccer league of RoboCup using the half field offense task.

References

[1]

Akiyama, H. 2010. Agent2d base code release. http://sourceforge.jp/projects/rctools.

[2]

Albus, J. S. 1971. A theory of cerebellar function. Mathematical Biosciences 10(12):25-61.

[3]

Albus, J. S. 1975. A new approach to manipulator control cerebellar model articulation control (cmac). Transactions on ASME, J. of Dynamic Systems, Measurement, and Control 97(9):220-227.

[4]

Barrett, S.; Stone, P.; Kraus, S.; and Rosenfeld, A. 2013. Teamwork with limited knowledge of teammates. In AAAI.

[5]

Blum, A., and Mansour, Y. 2007. Algorithmic Game Theory. Cambridge University Press. chapter Learning, regret minimization, and equilibria.

[6]

Bowling, M., and McCracken, P. 2005. Coordination and adaptation in impromptu teams. In Proceedings of the Nineteenth Conference on Artificial Intelligence (AAAI), 53-58.

[7]

Brafman, R. I., and Tennenholtz, M. 1996. On partially controlled multi-agent systems. Journal of Artificial Intelligence Research (JAIR) 4:477-507.

[8]

Conitzer, V., and Sandholm, T. 2007. AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Machine Learning (MLJ) 67.

[9]

Decker, K. S., and Lesser, V. R. 1995. Designing a family of coordination algorithms. In International Conference on Multi-Agent Systems (ICMAS), 73-80.

[10]

Deisenroth, M. P.; Neumann, G.; and Peters, J. 2013. A survey on policy search for robotics. Foundations and Trends in Robotics 2(1-2):1-142.

[11]

Doshi, P., and Zeng, Y. 2009. Improved approximation of interactive dynamic influence diagrams using discriminative model updates. In Proceedings of the Eighth International Conference on Autonomous Agents and Multiagent Systems (AAMAS).

[12]

Ernst, D.; Geurts, P.; and Wehenkel, L. 2005. Tree-based batch mode reinforcement learning. In Journal of Machine Learning Research (JMLR), 503-556.

[13]

Gal, Y., and Pfeffer, A. 2008. Network of influence diagrams: Reasoning about agents' beliefs and decision-making processes. Journal of Artificial Intelligence Research (JAIR) 33:109-147.

[14]

Gmytrasiewicz, P. J., and Doshi, P. 2005. A framework for sequential planning in multi-agent settings. Journal of Artificial Intelligence Research (JAIR) 24(1):49-79.

[15]

Grosz, B., and Kraus, S. 1996a. Collaborative plans for complex group actions. Artificial Intelligence (AIJ) 86:269-368.

[16]

Grosz, B. J., and Kraus, S. 1996b. Collaborative plans for complex group action. Artificial Intelligence (AIJ) 86(2):269-357.

[17]

Horling, B.; Lesser, V.; Vincent, R.; Wagner, T.; Raja, A.; Zhang, S.; Decker, K.; and Garvey, A. 1999. The TAEMS White Paper.

[18]

Jones, E.; Browning, B.; Dias, M. B.; Argall, B.; Veloso, M. M.; and Stentz, A. T. 2006. Dynamically formed heterogeneous robot teams performing tightly-coordinated tasks. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 570-575.

[19]

Kalyanakrishnan, S.; Liu, Y.; and Stone, P. 2007. Half field offense in RoboCup soccer: A multiagent reinforcement learning case study. In RoboCup-2006: Robot Soccer World Cup X, volume 4434 of Lecture Notes in Artificial Intelligence. Berlin: Springer Verlag. 72-85.

[20]

Liemhetcharat, S., and Veloso, M. 2012. Modeling and learning synergy for team formation with heterogeneous agents. In Proceedings of the Eleventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS).

[21]

MacAlpine, P.; Genter, K.; Barrett, S.; and Stone, P. 2014. The RoboCup 2013 drop-in player challenges: Experiments in ad hoc teamwork. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]

Stone, P.; Kaminka, G. A.; Kraus, S.; and Rosenschein, J. S. 2010. Ad hoc autonomous agent teams: Collaboration without pre-coordination. In Proceedings of the Twenty-Fourth Conference on Artificial Intelligence (AAAI).

[23]

Sutton, R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press.

[24]

Tambe, M. 1997. Towards flexible teamwork. Journal of Artificial Intelligence Research (JAIR) 7:83-124.

[25]

Valtazanos, A., and Ramamoorthy, S. 2013. Bayesian interaction shaping: Learning to influence strategic interactions in mixed robotic domains. In Proceedings of the Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 63-70.

[26]

Watkins, C. J. C. H. 1989. Learning from Delayed Rewards. Ph.D. Dissertation, King's College, Cambridge, UK.

[27]

Wu, F.; Zilberstein, S.; and Chen, X. 2011. Online planning for ad hoc autonomous agent teams. In The 22th International Joint Conference on Artificial Intelligence (IJCAI).

Cited By

Alkoby SRath AStone PElkind EVeloso MAgmon NTaylor M(2019)Teaching Social Behavior through Human Reinforcement for Ad hoc Teamwork - The STAR FrameworkProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3306127.3331914(1773-1775)Online publication date: 8-May-2019
https://dl.acm.org/doi/10.5555/3306127.3331914
Da Silva FCosta A(2019)A survey on transfer learning for multiagent reinforcement learning systemsJournal of Artificial Intelligence Research10.1613/jair.1.1139664:1(645-703)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1613/jair.1.11396
Golpayegani FDusparic IClarke S(2019)Using Social Dependence to Enable Neighbourly Behaviour in Open Multi-Agent SystemsACM Transactions on Intelligent Systems and Technology10.1145/331940210:3(1-31)Online publication date: 22-Apr-2019
https://dl.acm.org/doi/10.1145/3319402
Show More Cited By

Index Terms

Cooperating with unknown teammates in complex domains: a robot soccer case study of ad hoc teamwork
1. Computing methodologies
  1. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Local navigation-like functions for safe robot navigation in bounded domains with unknown convex obstacles
Abstract
In this paper, we propose a controller that stabilizes a holonomic robot with single-integrator dynamics to a target position in a bounded domain, while preventing collisions with convex obstacles. We assume that the robot can measure its own ...
A sub goal seeking approach for reactive navigation in complex unknown environments

Reactive-based approaches are widely used in autonomous navigation. However, in complex unknown environments, pure reactive-based navigation still poses a few challenges since it can be easily trapped by a local minimum and may produce some extra ...
Learning to Cooperate with Completely Unknown Teammates
Progress in Artificial Intelligence
Abstract
A key goal of ad hoc teamwork is to develop a learning agent that cooperates with unknown teams, without resorting to any pre-coordination protocol. Despite a vast number of ad hoc teamwork algorithms in the literature, most of them cannot address ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'15: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence

January 2015

4331 pages

ISBN:0262511290

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 25 January 2015

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alkoby SRath AStone PElkind EVeloso MAgmon NTaylor M(2019)Teaching Social Behavior through Human Reinforcement for Ad hoc Teamwork - The STAR FrameworkProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3306127.3331914(1773-1775)Online publication date: 8-May-2019
https://dl.acm.org/doi/10.5555/3306127.3331914
Da Silva FCosta A(2019)A survey on transfer learning for multiagent reinforcement learning systemsJournal of Artificial Intelligence Research10.1613/jair.1.1139664:1(645-703)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1613/jair.1.11396
Golpayegani FDusparic IClarke S(2019)Using Social Dependence to Enable Neighbourly Behaviour in Open Multi-Agent SystemsACM Transactions on Intelligent Systems and Technology10.1145/331940210:3(1-31)Online publication date: 22-Apr-2019
https://dl.acm.org/doi/10.1145/3319402
Rizk YAwad MTunstel E(2019)Cooperative Heterogeneous Multi-Robot SystemsACM Computing Surveys10.1145/330384852:2(1-31)Online publication date: 9-Apr-2019
https://dl.acm.org/doi/10.1145/3303848
Metcalf KTheobald BApostoloff NAndre EKoenig SDastani MSukthankar G(2018)Learning Sharing Behaviors with Arbitrary Numbers of AgentsProceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3237383.3237882(1232-1240)Online publication date: 9-Jul-2018
https://dl.acm.org/doi/10.5555/3237383.3237882
Chakraborty MChua KDas SJuba B(2017)Coordinated versus decentralized exploration in multi-agent multi-armed banditsProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3171642.3171667(164-170)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3171642.3171667
Albrecht SStone PLarson KWinikoff MDas SDurfee E(2017)Reasoning about Hypothetical Agent Behaviours and their ParametersProceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems10.5555/3091125.3091206(547-555)Online publication date: 8-May-2017
https://dl.acm.org/doi/10.5555/3091125.3091206
Liemhetcharat SVeloso M(2017)Allocating training instances to learning agents for team formationAutonomous Agents and Multi-Agent Systems10.1007/s10458-016-9355-331:4(905-940)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1007/s10458-016-9355-3
Hernandez-Leal PZhan YTaylor MSucar LMunoz De Cote E(2017)Efficiently detecting switches against non-stationary opponentsAutonomous Agents and Multi-Agent Systems10.1007/s10458-016-9352-631:4(767-789)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1007/s10458-016-9352-6
Amir OGrosz BGajos K(2016)Mutual influence potential networksProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence10.5555/3060621.3060732(796-803)Online publication date: 9-Jul-2016
https://dl.acm.org/doi/10.5555/3060621.3060732
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents