Hearthstone helper - using optical character recognition techniques for cards detection

Abstract

In this paper we address the problem of capturing, processing and analyzing images from the video stream of the Hearthstone game in order to obtain relevant information on the conduct of parties in this game. Since the information needs to be presented to the user in real-time, we needed to find the most suitable methods of extracting this information. Therefore, techniques such as background subtraction, histograms comparisons, key points matching, optical character recognition were investigated. Driven by the required processing speed, we ended up using optical character recognition on limited areas of interest from the captured image. After developing the application, we tested it in real-world context, while real games were played and presented the obtained results. In the end, we also provided two examples where the application would prove useful for better decision making during the game.