Evolving behaviour trees for automated discovery of novel combat strategy in real-time strategy wargames

Abstract

A wargame is defined as: "A simulation, by whatever means, of a military operation involving two or more opposing forces using rules, data, and procedures designed to depict an actual or assumed real life situation" (Gortney 2016, p503). Wargaming is used for several purposes, such as teaching strategic planning, practising the tasks associated with war, and analytic purposes (Burns et al., 2015). While wargaming is predominantly a human centric analytical activity, it is an area where artificial intelligence (AI) may play a useful role because of a computer's ability to play through a wider range of possible strategies. However, creating simulated AI participants for wargame scenarios is challenging because of the complexity and uncertainty in the environments in which they exist. The recent rise of automated behaviour discovery for agents in a variety of games traditionally dominated by humans offers a possible solution. Commercial real-time strategy (RTS) games provide an abstract simulation of a world where players aim to dominate and defeat other players by acquiring, using and managing resources, often including a mix of military, political, scientific and economic factors. As there is considerable overlap between the objectives found in RTS games to those that exist in military style wargames, RTS games are ideal platforms to conduct research and development in support of our AI-enabled wargaming research objectives. In this paper, we document our work on the use of evolutionary algorithms for automated behaviour discovery, where evolved behaviour trees are used as controllers for the blue team entities in the wargaming simulation. As behaviour trees are constructs formulated as a tree-type graph data structure, genetic programming (Koza, 1992), a technique developed specifically for the evolution of such structures was employed. In order to systematically evaluate the approach in terms of novel behaviour discovery, two test scenarios were designed to isolate particular features of land-based combat, inspired by terrain design patterns from computer games. A more complex scenario, involving multiple terrain constructs was also evaluated. A set of experiments were run on the developed scenarios to evolve a blue team against a static red team opponent. The red entities employed a reactive AI, with the simplicity of the red AI balanced by having a much greater number of red units in the scenario. Results from the experiments indicate that evolved behaviour tree controllers in a multi-agent scenario can be useful to identify a set of behaviours that exploit the properties of the scenario and lead to victory. In particular, we observed the 'expected' behaviour in the control scenarios: • Units learned to work together as a team, for example, armor units shielding artillery units. • Units in a relatively weak blue side learned to exploit a chokepoint in the terrain to defeat a superior red side. • Units learned to avoid dangerous sections of terrain, such as those protected by enemy snipers.