Evolving diverse strategies through combined phenotypic novelty and objective function search

Abstract

Novelty search is an algorithm which proposes open-ended exploration of the search space by maximising behavioural novelty, removing the need for an objective fitness function. However, we show that when applied to complex tasks, training through novelty alone is not sufficient to produce useful controllers. Alongside this, the definition of phenotypic behaviour significantly effects the strategies of the evolved solutions. Controller networks for the spaceship in the arcade game Asteroids were evolved with five different phenotypic distance measures. Each of these phenotypic measures are shown to produce controllers which adopt different strategies of play than controllers trained through standard objective fitness. Combined phenotypic novelty and objective fitness is also shown to produce differing strategies within the same evolutionary run. Our results demonstrate that for domains such as video games, where a diverse range of interesting behaviours are required, training agents through a combination of phenotypic novelty and objective fitness is a viable method.