Coordinated control of multi-agent teams is an important task in many real-time strategy (RTS) games. In most prior work, micromanagement is the commonly used strategy whereby individual agents operate independently and make their own combat decisions. On the other extreme, some employ a macromanagement strategy whereby all agents are controlled by a single decision model. In this paper, we propose a hierarchical command and control architecture, consisting of a single high-level and multiple low-level reinforcement learning agents operating in a dynamic environment. This hierarchical model enables the low-level unit agents to make individual decisions while taking commands from the high-level commander agent. Compared with prior approaches, the proposed model provides the benefits of both flexibility and coordinated control. The performance of such hierarchical control model is demonstrated through empirical experiments in a real-time strategy game known as StarCraft: Brood War (SCBW).

多智能体团队的协调控制是许多实时战略 (RTS) 游戏中的一项重要任务。在大多数先前的工作中，微观管理是常用的策略，个体代理独立运作并做出自己的战斗决定。在另一个极端，有些采用宏观管理策略，即所有代理都由一个决策模型控制。在本文中，我们提出了一种分层命令和控制架构，由在动态环境中运行的单个高级和多个低级强化学习代理组成。这种分层模型使低级单位代理能够在接受来自高级指挥官代理的命令的同时做出单独的决定。与先前的方法相比，所提出的模型提供了灵活性和协调控制的好处。