Classification of Icon Type and Cooldown State in Video Game Replays

Abstract

The potential to positively influence research developments in seemingly unrelated areas leads to an increasing interest in the analysis of video games. As game publishers rarely provide an open interface to gain access to in-game information, the proposed system relies on the availability of video game recordings and broadcasts and operates completely in the visual domain. The classification of video game icons and associated metadata serves as an example task to assess the potential of several image recognition methods, including Random Forests (RFs), Support Vector Machines (SVMs), and Convolutional Networks (ConvNets). The experiments show that all machine learning approaches are able to successfully classify game icons in their original state, but performance is significantly decreased for icons in a cooldown state. SVMs fail to estimate the correct cooldown state, while RFs are outperformed by ConvNets.