Planning and Execution Control Architecture for Infantry Serious Gaming

Alexandre Menif, Christophe Guettier1 and Tristan Cazenave2

1
SAGEM, 27, Rue Leblanc, 75012 Paris, France
2
LAMSADE, Université Paris-Dauphine, Paris, France
{alexandre.menif, christophe.guettier}@sagem.com
cazenave@lamsade.dauphine.fr

Abstract or military action. Using search techniques, automatic plan

generation can be used at different hierarchical levels to sim-
Serious gaming is developing among all modern armies for
ulate both the command chain and soldier activities.
teaching and training as well as for developing new concepts
of engagement. To reach a realistic level of simulation, on- The paper exhibits a planning and execution control ar-
line planning techniques provide an expressive and construc- chitecture that integrates mission management along the
tive approach to define basic tactical activities. To achieve a command-chain, automatic tactical sequence generation and
mission goal, a virtual soldier must follow a short-term plan execution control. The architecture makes use of Hierar-
that can be quickly reprocessed in order to follow changes chical Task Networks (HTN) representations, that facilitate
in the environment, orders or situation awareness. This pa- command-chain modelling, automatic goal breakdown as
per presents a planning and execution control architecture well as sequential task synthesis. To match mission exe-
for simulating the behaviour of virtual infantry soldier. The cution tempo and contingencies, the planning functionality
planning approach relies on the frequent generation of short can generate very short plans over small horizons. Once a
plans using a Hierarchical Task Networks approach. Execu-
tion control handles synchronisations of soldiers and trigger
plan is generated, each action is tentatively executed by the
replanning whenever action cannot be executed in simulation. execution controler which interacts with the simulated en-
Applied to two generic types of action, preliminary results vironment. Execution control also handles synchronisations
show that response times match the level of reactivity needed of soldiers and trigger replanning whenever action cannot be
for serious gaming. executed.
Applied to two generic types of action, preliminary re-
sults show that response times match the level of reactivity
Introduction needed for serious gaming.
Automatic planning has always given major challenges in First section focuses on the relation between serious gam-
defence domains. Many problem models and search tech- ing and military requirements for infantry warfare at tactical
niques have been considered at strategic, operative or tacti- level. The second section describes a software planning ar-
cal command levels. However, simulation of low level tac- chitecture intended to meet those requirements. The next
tics and basic soldier behaviours refer in general to action section presents the modelling of a planning domain for in-
scripting. In order to match the expected level of realism fantry tactical behaviours and provide details about a new
required by modern armies, this practice tends to become a implementation of SHOP2. Finally the last parts are dedi-
tremendous and time-consuming engineering activity. cated to the state of the art and conclusion.
In video game, however, planning techniques have be-
come more and more popular for a decade. The experience Soldier Level Serious Gaming
of planning in video games has started with FEAR, a first
person shooter issued in 2005, which implements an algo- Rather than using old fashion tactical simulation, serious
rithm called GOAP (Goal Oriented Action Planning) to pro- gaming is a very promising approach to teach, train and de-
vide an efficient coordinated behaviour to the enemies. Al- velop new concepts of engagement. Serious gaming reuses
though GOAP was inspired on STRIPS, one of the oldest video game design and offers immersive and interactive en-
algorithm in actions planning, this experience has success- vironment to armies end-users. To be effective in profes-
fully illustrated the possibility to compute short linear plans sional tasks, these environments necessitate realistic and in-
for a few coordinated agents in real time (Orkin 2006). telligent behaviour for simulated soldiers.
Planning techniques provide an expressive way to model Different requirements stress both agents behaviour as
tactical behaviour, by developing a compositional approach well as the architectural design of such serious gaming:
to basic activities. Several dedicated planning domains can
• Teaching: Basic infantry scenarii must be easy to pro-
be developed according to the type of mission, environment
gram, and soldier behaviour must be realistic enough to
Copyright © c 2013, Association for the Advancement of Artificial stimulate the end-user. Basic action sequences can be
Intelligence (www.aaai.org). All rights reserved. analysable to give explanations during after action review.

31
• Training: Such training stresses coordination, orders and
reporting capabilities of the end-user. The serious game
must be able to handle a large set of soldiers, and to con-
struct complex situation. Again, in spite of their complex-
ity, user performances can be analysed during after action
review.
• Concept development: the development of new opera-
tional concept or the integration of new systems need doc-
trinal evolution. On one hand, serious gaming can help
understanding the impact of new tactics, operational tech-
niques and procedures. On the other hand, it provides a
strong support to evaluate new system performances and
man-machine interactions.

Planning and Execution Control Architecture Figure 1: Global planning architecture for agent gaming
Figure 1 gives an overview of the Planning and Execution
Control Architecture. It features two different levels of plan- squad platoon company battalion
ning functionalities. Mission management solves medium
timeline < second < 5 mins 5 to 15 mins > hour
and long term planning from brigade down to the platoon units 10 4 16 70
level. Below this level, squads and soldier sequence of ac-
tions are generated by a dedicated HTN planner. This core
component drives the quality of virtual soldier behaviour Figure 2: This table gives the timeliness with respect to the
and is detailed in a dedicated section. number of units to plan for
The planning architecture is meant to be integrated in a
real time simulator, which means that it is not possible to
spend much processing power in planning procedures that a plan) with an associated schedule from the initial position
may alter frame rate, gameplay and graphical rendering. The to the mission objective. The mission planning problem can
architecture must scale up to several dozens of entities to be be modelled and solved following a complete constraint pro-
managed in parallel, potentially leading to many planning gramming approach (Guettier 2007).
queries to be conducted simultaneously (see figure 2). The mission manager must also handle plan breakdown
Both planners takes inputs from a situation awareness and along the tactical command chain. Each hierarchical level
threat assessment components that gathers and fuses data defines an operational order (OPORD) using its own mission
from the simulation environment. The execution control goal. The OPORD tasks / organises units of a lower eche-
component processes the realisation of a sequence of actions lon and allocates resources. This part is not automated yet,
in the simulator kernel. Whenever a basic sequence of action since many constructive parameters have to be taken in ac-
cannot be correctly executed in the simulation environment, count simultaneously (organisational, logistics, assessment
a replanning event is triggered. of enemy situation and associated course of action, complex
Mission manager, situation awareness, threat assessment coordination procedures...). However, it is possible to auto-
and execution control modules are fundamental software matically structure the different OPORD by using the out-
components for the architecture design, however their de- come of both mission planner and units organisation.
tailed design are out of the scope of this paper. When major changes occur during mission execution,
global replanning might be necessary. This has two impacts:
Mission manager • A new problem instance is provided to the planner. Then,
The mission manager combines mission planning and plan repair or local search can be used to find a plan that
scheduling as well as plan decomposition for lower units. cope with the new situation.
Given initial conditions (on both friendly and enemy units), • OPORD are updated, using so-called FRAGmentary Or-
mission planning and scheduling (P&S) defines the course ders (FRAGO).
of actions to reach mission objectives. P&S takes in account
terrain structure, unit capabilities and their coordination : Execution Controller
• Terrain representation involve axis of advances, observa- The execution controller executes action provided by the se-
tion points, covers and concealments. quence generator. In several ways it interacts with the sim-
• Capabilities refer to mobility, engagement, communica- ulation environment by controlling the virtual soldier. For
tion and observations. a given action, it will set the virtual soldier mobility, ori-
entation, and posture. Through the simulator, the controller
• Coordination of actions is needed in time and space for can request a waypoint to reach, assess visibility of a line-
self protection, or to reach an expected effect. of-sight, or find threatening objects within its field of view.
The problem solved by the mission manager P&S is to To evaluate the success of an action, the execution controller
find, for each unit, a course of actions and movements (e.g. uses the following logic:

32
 ; monitor problem definition
• Preconditions are verified. Their scope are mobility (way (defproblem problem monitor
; this line describe the environment as a set of predicates
points), previous action termination, observation results, ((down soldier1) (detected soldier1 target sector))
or event occurring in the environment. ; the task assign to the soldier
((monitor soldier1 sector)))
• Request to the simulator succeeds (for instance, the path
; a computed plan for the monitor problem
finder has been able to find a waypoint). (:task !report soldier1 target)
(:task !stand-up soldier1)
• Time / space coordination can be met. These coordination (:task !use-weapon soldier1 target)
can be either defined by the mission planner or by collab- ; patrol problem definition
orative type of actions (for instance a mutual protection or (defproblem problem patrol
((sector front) (sector sector1) (unsafe sector1) (front fireteam soldier2)
a synchronised mobility action). (back fireteam soldier1) (covering soldier1 front) (covering soldier2 sect
(up soldier1) (up soldier2))
Whenever an action cannot be successfully achieved, a ((patrol fireteam)))
replanning event is triggered. ; a computed plan for the patrol problem
(:task !cover soldier2 sector1)
(:task !cover soldier1 front)
Situation awareness and threat assessment (:task !find-cover-point soldier1 sector1)
(:task !go-to-cover-point soldier1 sector1)
Situation awareness is maintained by a Red Force Tracker (:task !pass soldier1 soldier2)
(:task !cover soldier1 sector1)
(RFT) directly inspired from target acquisition and tracking
systems (Sella & al. 2011). The RFT service tackles ob-
servation reports, target association, short term position es- Figure 3: Planning request and plan answer for monitoring
timate, long term enemy course of action prediction. Data actions
fusion algorithms such as Kalman Filtering (KF), Interac-
tive Multiple Models, or Multiple Hypothesis Tracking are
integrated to associate observations, remove inconsistencies, Infantry domain modelling
and to manage an active list of tracks. Delayed reporting and It sounds crucial that the lower a unit is in the command
tracking are also simulated such that two units does not nec- chain, the simpler its behaviour should be defined. For ex-
essarily have the same immediate awareness (note that in ample, a single soldier would probably be constantly plan-
real life, this knowledge is actuated by issuing an OPORD ning to adapt his activity to a constantly changing environ-
through the chain of command). ment, so his behaviour should be extremely cheap to plan.
Based on these outputs, whenever the enemy situation Therefore, our requirement for a planning domain relies on
changes, each soldier or squad evaluates its own threat level two aspects: actions at one hierarchical level should only
in order to potentially recompute a new plan sequence. focus on its level and the synchronization of the direct sub
level, and at the bottom of the hierarchy, planning should be
Tactical sequence generator extremely simple. With those rules in mind, we have started
Each hierarchical unit uses planning for its level and then to design a simple planning domain that could match our
assigns orders for its sub-level units. Units coordination is expectation, using SHOP formalism (Nau & al. 2003).
verified by planning at the upper level, so that each level The planning domain (see appendix) describes a set of
can behave more autonomously within its own action area. very simple actions for monitoring and patrolling tasks, car-
It also results in time / space synchronisations, enforced by ried out by one soldier or a two-soldiers fireteam. The ”mon-
the execution controller. itor” action illustrates how planning would interact with
In combat simulation, the tactical environment is con- events from the simulator. Here the simulator would request
stantly evolving, so that plans can be quickly invalidated. planning for the ”monitor” task each time an event modifies
Replanning is necessary to comply with the up to date situa- the situation (in this case, simply either there is an enemy
tion awareness. Relying on a planning domain where actions which is detected or not). Then, the virtual soldier has a
would be interleaved between hierarchical levels would be few possible behaviours. He may or may not engage the tar-
hard to manage. Indeed this would likely produce complex get, according to its ability to do it or if it is allowed to do
and expensive plans that would be compromised by the first it. The ”patrol” action is collaborative (it involves two sol-
unexpected event. diers from a fireteam) and handles the coordination between
Instead, the proposed approach is closed to the Com- the two soldiers (so one soldier only moves if it is covered
mand Hierarchies concept (Pittman 2008). This results in by the other), and will assign very simple tasks to each sol-
a much more flexible way of planning: not only the global dier agent. Figure 3 representes two typical queries for both
plan search complexity would be reduced, but an unexpected examples (planning requests for actions ”monitor” and ”pa-
event would only affect a tiny sub-part of the overall plan trol”) as they would be issued by the simulated environment,
and not trigger a re-planning for the whole operation. An- and the replies from the planning system: two plans as se-
other aspect is that low-levels units would probably be much quences of tasks.
more subjected to planning events than high-levels ones. For
example, the detection of obstacles, traps or enemies would Search Algorithm
surely alter the way a fireteam had planned to conduct its In order to constitute a benchmark of popular planning al-
mission, but not necessarily the global manoeuvre of a pla- gorithms, a first implementation of SHOP2 is under devel-
toon. opment. The C++ programming language is used as it is

33
the one employed in the targeted simulator product. At its ((up ?soldier))
((down ?soldier)))
current state, the planner implements the main core func-
tionalities of the original one from Nau (Nau & al. 2003), (:operator (!stand-up ?soldier)
((down ?soldier))
alongside with a parser component that is able to read plan- ((down ?soldier))
ning domains and problems written with a subset of the for- ((up ?soldier)))

malism described for SHOP. (:operator (!cover ?soldier1 ?sector1)

(and (sector ?sector2) (covering ?soldier1 ?sector2))
On different situations, all the previously detailed exam- ((covering ?soldier1 ?sector2))
ples (figures 3) are computed in less than a millisecond with ((covering ?soldier1 ?sector1)))
our current implementation of SHOP2. Even if actual plan- (:operator (!follow ?soldier1 ?soldier2) () () ())
ning domains would probably be more sophisticated, this is
(:operator (!go-to-waypoint ?soldier1) ((have-waypoint ?soldier1)) () ())
the target time we will have to achieve for such basic level
hierarchical units. (:operator (!find-cover-point ?soldier1 ?sector1)
()
()
Conclusion ((have-cover-point ?soldier1 ?sector1)))

(:operator (!go-to-cover-point ?soldier1 ?sector1)

This paper proposed a way to apply planning techniques ((have-cover-point ?soldier1 ?sector1))
from the video games experience on basic military units be- ()
((at-cover-point ?soldier1 ?sector1)))
haviour to professional simulation tools. The main objec-
tive is to demonstrate that tactical behaviours for low level (:operator (!pass ?soldier1 ?soldier2)
(and (front ?fireteam ?soldier2) (back ?fireteam ?soldier1))
entities of military hierarchy (individual soldier, fireteam, ((front ?fireteam ?soldier2) (back ?fireteam ?soldier1))
((front ?fireteam ?soldier1) (back ?fireteam ?soldier2)))
squad...) could be encoded and executed with low computa-
tional power through two main considerations. (:method (use-weapon ?soldier ?target)
; soldier cannot use weapon if down
On one hand, hierarchical planners seem adapted to ((down ?soldier))
model complex behaviour for coordinated units. They af- ((!stand-up ?soldier) (!use-weapon ?soldier ?target))
; soldier already up
fords a segregation of the chain of command from the lo- ()
cal virtual agent behaviour, alongside with a convenient ex- ((!use-weapon ?soldier ?target)))

pression of action sequences. On the other hand, a reactive (:method (watch ?soldier ?area)
; soldier cannot watch if down
architecture, aware of any triggered events from the simu- ((down ?soldier))
lated environment, can restrict the time horizon for planning ((!stand-up ?soldier) (!watch ?soldier ?area))
; soldier already up
request considerably, and thus reduces the need for compli- ()
cated planning processes. ((!watch ?soldier ?area)))

The dedicated planner fits easily with other components (:method (engage ?soldier ?target)
(long term mission planning, situation awareness and execu- ; soldier must protect itself
((under-fire ?soldier))
tion control). In terms of software engineering, the approach ((!bend-down ?soldier))
; target is neither hostile nor can be engaged
provides a strong gain compared to scripting methods. ; according to rule of engagement
((not (hostile ?target)) (not (can-engage ?soldier ?target)))
((!report ?soldier ?target) (use-weapon ?soldier ?target))
References ; else engage target
()
Guettier C. Solving Planning and Scheduling Problems in Net- ((use-weapon ?soldier ?target)))
work based Operations. Proceedings of CP’07, USA. 2007.
(:method (monitor ?soldier ?area)
Sella, G.; Cherrier, O.; Guettier, C. & Yelloz, J. Development and ; a target is detected while monitoring
Experimentation of Collaborative Red Force Tracking in Service (and (detected ?soldier ?target ?area))
((engage ?soldier ?target))
Oriented Architecture for Tactical Networking Systems Procees- ; else, keep watching
dings of MILCOM’11, USA. 2011 ()
((watch ?soldier ?area)))
Orkin, J.. Three states and a plan: the AI of FEAR. In Game
Developers Conference. 2006. (:method (patrol ?fireteam)
; if a target is detected in a given sector, engage it
Nau, D. S.; Au, T. C.; Ilghami, O.; Kuter, U.; Murdock, J. W.; ((detected ?target ?sector1) (covering ?soldier1 ?sector1))
((engage ?soldier1 ?target))
Wu, D. & Yaman, F. SHOP2: An HTN planning system. J. Artif. ; if there is an unsafe sector, use parrot-like move
Intell. Res. (JAIR), 20, 379-404. 2003. ((sector ?sector1) (unsafe ?sector1)
(front ?fireteam ?soldier1) (back ?fireteam ?soldier2))
Pittman, D. Command Hierarchies Using Goal-Oriented Action ((!cover ?soldier1 ?sector1) (!cover ?soldier2 front)
Planning, AI Game Wisdom 4, Charles River Media (2008), (!find-cover-point ?soldier2 ?sector1)
(!go-to-cover-point ?soldier2 ?sector1)
pages 383 to 391. 2008. (!pass ?soldier2 ?soldier1) (!cover ?soldier2 ?sector1))
; else progress normally
((have-waypoint ?soldier1) (front ?fireteam ?soldier1)
Planning domain (back ?fireteam ?soldier2))
((!cover ?soldier1 front) (!cover ?soldier2 far)
(!follow soldier2 soldier1) (!go-to-waypoint ?soldier1)))))
(defdomain monitor (
(:operator (!use-weapon ?soldier ?target) ((up ?soldier)) () ())

(:operator (!watch ?soldier ?area) ((up ?soldier)) () ())

(:operator (!report ?soldier ?target) () () ((can-engage ?soldier ?target)))

(:operator (!bend-down ?soldier)

((up ?soldier))

34