research-article

A combined tactical and strategic hierarchical learning framework in multi-agent games

Authors:

Ho-lun ChengAuthors Info & Claims

Sandbox '08: Proceedings of the 2008 ACM SIGGRAPH symposium on Video games

Pages 115 - 122

https://doi.org/10.1145/1401843.1401865

Published: 09 August 2008 Publication History

Abstract

This paper presents a novel approach to modeling a generic cognitive framework in game agents to provide tactical behavior generation as well as strategic decision making in modern multi-agent computer games. The core of our framework consists of two characterization concepts we term as the tactical and strategic personalities, embedded in each game agent. Tactical actions and strategic plans are generated according to the weights defined in their respective personalities. The personalities are constantly improved as the game proceeds by a learning process based on reinforcement learning. Also, the strategies selected at each level of the agents' command hierarchy affect the personalities and hence the decisions of other agents. The learning system improves performance of the game agents in combat and is decoupled from the action selection mechanism to ensure speed. The variability in tactical behavior and decentralized strategic decision making improves realism and increases entertainment value. Our framework is implemented in a real game scenario as an experiment and shown to outperform various scripted opponent team tactics and strategies, as well as one with a randomly varying strategy.

References

[1]

Andrade, G., Ramalho, G., Santana, H., and Corruble, V. 2005. Automatic computer game balancing: A reinforcement learning approach. In Proceedings of the Autonomous Agents And Multi Agent Systems Conference (July), 1111, 1112.

Digital Library

[2]

Blizzard Entertainment, 2006. Warcraft III. Accessed April 12, 2006. Available via http://www.blizzard.com/war3/.

[3]

Blizzard, 2006. Diablo ii. Accessed April 12, 2006. Available via http://www.blizzard.com/diablo2/.

[4]

Blizzard, 2006. World of warcraft. Accessed April 12, 2006. Available via http://www.worldofwarcraft.com/.

[5]

Buro, M., 2007. ORTS - A Free Software RTS Game Engine. Accessed March 20, 2007. Available via http://www.cs.ualberta.ca/mburo/orts/index.html.

[6]

Charles, D., Kerr, A., McNeill, M., McAlister, M., Black, M., Kcklich, J., Moore, A., and Stringer, K. 2005. Player-centred game design: Player modeling and adaptive digital games. In Proceedings of the Digital Games Research Conference, 285,298.

[7]

Christian J. Darken, G. H. P. 2006. Findin Cover in Dynamic Environments, first ed. Charles River Media, Hingham, Massachusetts.

[8]

Co-op, S., 2006. Sven co-op. Accessed September 15, 2006. Available via http://www.svencoop.com/.

[9]

Electronic Arts, 2006. Battlefield 2142. Accessed December 20, 2006. Available via http://battlefield.ea.com/battlefield/bf2142/.

[10]

Exluna, Inc. 2002. Entropy 3.1 Technical Reference, January.

[11]

Fedkiw, R., Stam, J., and Jensen, H. W. 2001. Visual simulation of smoke. In Proceedings of SIGGRAPH 2001, ACM Press / ACM SIGGRAPH, E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 15--22.

Digital Library

[12]

Foka, A., and Trahanias, P. 2007. Real-time hierarchical POMDPs for autonomous robot navigation. In Proceedings of Robotics and Autonomous Systems, 561,571.

Digital Library

[13]

Geramifard, A., Chubak, P., and Bulitko, V. 2006. Biased cost pathfinding. In Proceedings of the Artificial Intelligence and Interactive Digital Entertainment conference, 112,114.

[14]

Horswill, I., and Zubek, R. 1999. Robot architectures for believable game agents. In Proceedings of the 1999 AAAI Spring Symposium on Artificial Intelligence and Computer Games, AAAI Technical Report SS-99-02.

[15]

Hunicke, R., and Chapman, V. 2004. AI for Dynamic Difficult Adjustment in Games. In Proceedings of the Challenges in Game AI Workshop, Nineteenth National Conference on Artificial Intelligence.

Digital Library

[16]

Hussain, T. S., and Vidaver, G. 2006. Flexible and purposeful npc behaviors using real-time genetic control. In Proceedings of The IEEE Congress on Evolutionary Computation (July), 785,792.

[17]

Jobson, D. J., Rahman, Z., and Woodell, G. A. 1995. Retinex image processing: Improved fidelity to direct visual observation. In Proceedings of the IS&T Fourth Color Imaging Conference: Color Science, Systems, and Applications, vol. 4, 124--125.

[18]

Kartch, D. 2000. Efficient Rendering and Compression for Full-Parallax Computer-Generated Holographic Stereograms. PhD thesis, Cornell University.

Digital Library

[19]

Khoo, A., and Dunham, G. 2002. Efficient, realistic npc control systems using behavior-based techniques. In AAAI Technical Report, 02--01.

[20]

Landis, H., 2002. Global illumination in production. ACM SIGGRAPH 2002 Course #16 Notes, July.

[21]

Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital michelangelo project. In Proceedings of SIGGRAPH 2000, ACM Press / ACM SIGGRAPH, New York, K. Akeley, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 131--144.

Digital Library

[22]

McDonald, D., Leung, A., Ferguson, W., and Hussain, T. 2006. An abstraction framework for cooperation among agents and people in a virtual world. In Proceedings of the Second Conference on Artificial Intelligence and Interactive Digital Entertainment (June).

[23]

Michael E. Tipping, C. M. B. 1999. Probabilistic principal component analysis. Technical Report NCRG/97/010, Neural Computing Research Grou (September).

[24]

NCsoft, 2006. Guild wars. Accessed April 12, 2006. Available via http://www.guildwars.com/.

[25]

Oliehoek, F. 2005. Game Theory and AI: A unified Approach to Poker Games. Masters Thesis, 561,571.

[26]

Orkin, J. 2004. Applying Goal-Oriented Action Planning to Games, first ed. Charles River Media, Hingham, Massachusetts.

[27]

Orkin, J. 2004. Finite State Machines, first ed. Charles River Media, Hingham, Massachusetts.

[28]

Orr, G., Schraudolph, N., and Cummins, F., 1999. Cs-449: Neural networks lecture notes. Accessed December 20, 2005. Available via http://www.willamette.edu/gorr/classes/cs449/intro.html.

[29]

Park, S. W., Linsen, L., Kreylos, O., Owens, J. D., and Hamann, B. 2006. Discrete sibson interpolation. IEEE Transactions on Visualization and Computer Graphics 12, 2 (Mar./Apr.), 243--253.

Digital Library

[30]

Parke, F. I., and Waters, K. 1996. Computer Facial Animation. A. K. Peters.

Digital Library

[31]

Pellacini, F., Vidimče, K., Lefohn, A., Mohr, A., Leone, M., and Warren, J. 2005. Lpics: a hybrid hardware-accelerated relighting engine for computer cinematography. ACM Transactions on Graphics 24, 3 (Aug.), 464--470.

Digital Library

[32]

Remco Straatman, A. B., and van der Sterren, W. 2006. Dynamic Tactical Postion Evaluation, first ed. Charles River Media, Hingham, Massachusetts.

[33]

Sailera, F., Buro, M., and Lanctot, M. 2007. Adversarial planning through strategy simulation. In Proceedings of the IEEE Symposium on Computational Intelligence and Games (April).

[34]

Sako, Y., and Fujimura, K. 2000. Shape similarity by homotropic deformation. The Visual Computer 16, 1, 47--61.

Digital Library

[35]

Sierra, 2006. No one lives forever 2. Accessed April 12, 2006. Available via http://nolf.sierra.com/.

[36]

Sierra, 2007. First encounter assault recon. Accessed December 23, 2007. Available via http://www.whatisfear.com/fear.html.

[37]

Silver, D. 2005. Cooperative pathfinding. In Proceedings of the First Artificial Intelligence and Interactive Digital Entertainment conference, 117,122.

Digital Library

[38]

Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. 2004. Difficulty Scaling of Game AI. In Proceedings of GAMEON 2004: 5th International Conference on Intelligent Games and Simulation, 33,37.

[39]

Spronck, P., Ponsen, M., Sprinkhuizen-Kuyper, I., and Postma, E. 2006. Adaptive Game AI with Dynamic Scripting. Springer Netherlands, Netherlands.

[40]

Spronck, P. 2005. A model for reliable adaptive game intelligence. In IJCAI-05 Workshop on Reasoning, Representation, and Learning in Computer Games, 95,100.

[41]

Sutton, R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, Massachusetts.

Digital Library

[42]

Sweetser, P. 2005. An emergent approach to game design. Ph.D Thesis. Available via http://www.itee.uq.edu.au/penny/publications.

[43]

Tan, C. T., and Cheng, H. 2007. Personality-based Adaptation for Teamwork in Game Agents. In Proceedings of the Third Conference on Artificial Intelligence and Interactive Digital Entertainment, 37.

[44]

Thue, D., and Bulitko, V. 2006. Modeling goal-directed players in digital games. In Proceedings of the Artificial Intelligence and Interactive Digital Entertainment conference, 285,298.

[45]

Wallace, N. 2004. Hierarchical Planning in Dynamic Worlds, first ed. Charles River Media, Hingham, Massachusetts.

[46]

White, C., and Brogan, D. 2006. The self organization of context for multi agent games. In Proceedings of 2nd Annual Conference on Artificial Intelligence in Interactive Digital Entertainment (June).

[47]

Wikipedia, 2006. Game balance. Accessed December 20, 2006. Available via http://en.wikipedia.org/wiki/Gamebalance.

[48]

Yannakakis, G. N., and Maragoudakis, M. 2005. Player modeling impact on players entertainment in computer games. In Springer-Verlag: Lecture Notes in Computer Science, 3538:74.

Digital Library

[49]

Yee, Y. L. H. 2000. Spatiotemporal sensistivity and visual attention for efficient rendering of dynamic environments. Master's thesis, Cornell University.

Cited By

Tan CCheng H(2018)Tactical agent personalityInternational Journal of Computer Games Technology10.1155/2011/1071602011(1-1)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2011/107160
Shi YCrawfis R(2014)Group tactics utilizing suppression and shelter2014 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES)10.1109/CGames.2014.6934139(1-8)Online publication date: Jul-2014
https://doi.org/10.1109/CGames.2014.6934139

Index Terms

A combined tactical and strategic hierarchical learning framework in multi-agent games

Recommendations

First-Person Shooter Game for Virtual Reality Headset with Advanced Multi-Agent Intelligent System
MM '16: Proceedings of the 24th ACM international conference on Multimedia

We present a multiplayer first-person shooter (FPS) game with advanced intelligent non-playable characters (NPC) under computer control. The game is specially adapted for playing in VR headset so the simulator sickness symptoms are significantly ...
Multi-agent reinforcement learning in games
Multi-agent learning in extensive games with complete information
AAMAS '03: Proceedings of the second international joint conference on Autonomous agents and multiagent systems

Learning in a multi-agent system is difficult because the learning environment jointly created by all learning agents is time-variant. This paper studies the model of multi-agent learning in complete-information extensive games (CEGs). We provide two ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Sandbox '08: Proceedings of the 2008 ACM SIGGRAPH symposium on Video games

August 2008

183 pages

ISBN:9781605581736

DOI:10.1145/1401843

Conference Chair:
David Schwartz
Rochester Institute of Technology
,
Program Chairs:
Karen Schrier
Scholastic/Columbia University
,
Chris Swain
USC School of Cinematic Arts
,
Michael Wagner
Danube University Krems

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Sandbox '08

Sponsor:

SIGGRAPH

Sandbox '08: An ACM SIGGRAPH Video Game Symposium

August 9 - 10, 2008

California, Los Angeles

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
711
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tan CCheng H(2018)Tactical agent personalityInternational Journal of Computer Games Technology10.1155/2011/1071602011(1-1)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2011/107160
Shi YCrawfis R(2014)Group tactics utilizing suppression and shelter2014 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES)10.1109/CGames.2014.6934139(1-8)Online publication date: Jul-2014
https://doi.org/10.1109/CGames.2014.6934139

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents