Abstract
Multi-Agent Learning (MAL) is a complex problem, especially in real-time systems where both cooperative and competitive learning are involved. We study this problem in the RoboCup Soccer Keepaway-Takeaway game and propose Argumentation Accelerated Reinforcement Learning (AARL) for this game. AARL incorporates heuristics, represented by arguments in Value-Based Argumentation, into Reinforcement Learning (RL) by using Heuristically Accelerated RL techniques. We empirically study for a specific setting of the Keepaway-Takeaway game the suitability of AARL, in comparison with standard RL and hand-coded strategies, to meet the challenges of MAL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bench-Capon, T.: Persuasion in practical argument using value-based argumentation frameworks. J. Log. Comput. 13(3), 429–448 (2003)
Bianchi, R.: Using heuristics to accelerate reinforcement learning algorithms. Ph.D. thesis, University of São Paulo (2004) (in Portuguese)
Bianchi, R., Ribeiro, C., Costa, A.: Accelerated autonumous learning by using heuristic selection of actions. Journal of Heuristics 14, 135–168 (2008)
Bradtke, S., Duff, M.: Reinforcement learning methods for continuous-time markov decision problems. Advances in Neural Information Processing Systems 7, 393–400 (1995)
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: The Proc. of AAAI (1998)
Devlin, S., Grzes, M., Kudenko, D.: An empirical study of potential-based reward shaping and advice in complex, multi-agent systems. Advances in Complex Systems 14, 251–278 (2011)
Dung, P.M.: On the acceptability of arugments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence 77(2), 321–357 (1995)
Fan, X., Toni, F.: Argumentation dialogues for two-agent conflict resolution. In: Proc. of COMMA (2012)
Ferretti, E., Errecalde, M., GarcÃa, A., Simari, G.: An application of defeasible logic programming to decision making in a robotic environment. In: LPNMR (2007)
Gao, Y., Toni, F., Craven, R.: Argumentation-based reinforcement learning for robocup soccer keepaway. In: Proc. of ECAI (2012)
Ghavamzadeh, M., Mahadevan, S., Makar, R.: Hierarchical multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems 13, 197–229 (2006)
Guestrin, C., Lagoudakis, M., Parr, R.: Coordinated reinforcement learning. In: Machine Learning International Workshop Then Conference (2002)
Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proc. of ICML (1998)
Iscen, A., Erogul, U.: A new perspective to the keepaway soccer: The takers (short paper). In: Proc. of AAMAS (2008)
Lau, Q.P., Lee, M.L., Hsu, W.: Coordination guided reinforcement learning. In: Proc. of AAMAS (2012)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proc. of ICML (1994)
Min, H.Q., Zeng, J.A., Chen, J., Zhu, J.H.: A study of reinforcement learning in a new multiagent domain. In: 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (2008)
Mozina, M., Zabkar, J., Bratko, I.: Argument based machine learning. Artificial Intelligence 171, 922–937 (2007)
Sen, S., Sekaran, M., Hale, J.: Learning to coordinate without sharing information. In: Proc. of AAAI (1994)
Singh, S.P., Sutton, R.S.: Reinforcement learning with replacing eligibility traces. Machine Learning 22, 123–158 (1996)
Stone, P., Sutton, R., Kuhlmann, G.: Reinforcement learning for robocup soccer keepaway. Adaptive Behavior 13, 165–188 (2005)
Sutton, R., Barto, A.: Reinforcement Learning. MIT Press (1998)
Tambe, M., Jung, H.: The benefits of arguing in a team. AI Magzine 20(4), 85–92 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, Y., Toni, F. (2014). Argumentation Accelerated Reinforcement Learning for RoboCup Keepaway-Takeaway. In: Black, E., Modgil, S., Oren, N. (eds) Theory and Applications of Formal Argumentation. TAFA 2013. Lecture Notes in Computer Science(), vol 8306. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54373-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-54373-9_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-54372-2
Online ISBN: 978-3-642-54373-9
eBook Packages: Computer ScienceComputer Science (R0)