CLASSQ-L: A Q-learning algorithm for adversarial real-time strategy games

Abstract

We present CLASSQ-L (for: class Q-learning) an application of the Q-learning reinforcement learning algorithm to play complete Wargus games. Wargus is a real-time strategy game where players control armies consisting of units of different classes (e.g., archers, knights). CLASSQ-L uses a single table for each class of unit so that each unit is controlled and updates its class' Q-table. This enables rapid learning as in Wargus there are many units of the same class. We present initial results of CLASSQ-L against a variety of opponents.