Combining case-based reasoning and reinforcement learning for unit navigation in real-time strategy game AI

Abstract

This paper presents a navigation component based on a hybrid case-based reasoning (CBR) and reinforcement learning (RL) approach for an AI agent in a real-time strategy (RTS) game. Spatial environment information is abstracted into a number of influence maps. These influence maps are then combined into cases that are managed by the CBR component. RL is used to update the case solutions which are composed of unit actions with associated fitness values. We present a detailed account of the architecture and underlying model. Our model accounts for all relevant environment influences with a focus on two main subgoals: damage avoidance and target approximation. For each of these subgoals, we create scenarios in the StarCraft RTS game and look at the performance of our approach given different similarity thresholds for the CBR part. The results show, that our navigation component manages to learn how to fulfill both sub-goals given the choice of a suitable similarity threshold. Finally, we combine both subgoals for the overall navigation component and show a comparison between the integrated approach, a random action selection, and a target-selection-only agent. The results show that the CBR/RL approach manages to successfully learn how to navigate towards goal positions while at the same time avoiding enemy attacks.