Using domain knowledge to improve Monte-Carlo tree search performance in parameterized poker squares

Abstract

Poker Squares is a single-player card game played on a 5 x 5 grid, in which a player attempts to create as many highscoring Poker hands as possible. As a stochastic single-player game with an extremely large state space, this game offers an interesting area of application for Monte-Carlo Tree Search (MCTS). This paper describes enhancements made to the MCTS algorithm to improve computer play, including pruning in the selection stage and a greedy simulation algorithm. These enhancements make extensive use of domain knowledge in the form of a state evaluation heuristic. Experimental results demonstrate both the general efficacy of these enhancements and their ideal parameter settings.