Graph Convolutional Networks for Turn-Based Strategy Games

Abstract

In this paper, we research Turn-Based Strategy (TBS) games that allow players to move multiple pieces in one turn and have multiple initial states. Compared to a game like Chess, which allows only one piece to move per turn and has a single initial state, it is difficult to create a strong computer player for such a group of TBS games. Deep learning methods such as AlphaZero and DQN are often used to create strong computer players. Convolutional neural networks (CNNs) are used to output policies and/or values, and input states are represented as “image”-like data. For TBS games, we consider that the relationships among units are more important than their absolute positions, and we attempt to represent the input states as “graphs”. In addition, we adopt graph convolutional neural networks (GCNs) as the suitable networks when inputs are graphs. In this research, we use a TBS game platform TUBSTAP as our test game and propose to (1) represent TUBSTAP game states as graphs, (2) employ GCNs as value network to predict the game result (win/loss/tie) by supervised learning, (3) compare the prediction accuracy of GCNs and CNNs, and (4) compare the playing strength of GCNs and CNNs when the learned value network is incorporated into a tree search. Experimental results show that the combination of graph input and GCN improves the accuracy of predicting game results and the strength of playing TUBSTAP.