Visual perception and analysis as first steps toward human–robot chess playing

Abstract

We propose in this paper a novel visual computing approach for the automatic perception of chess gaming states, where a standard chessboard with original chess pieces is used. Our image analysis algorithm uses only grayscale images captured from a single mobile camera for interactive gaming under natural environmental conditions. On the one hand, we apply computer vision techniques to detect and localize the grid corners, obtaining a 2D representation of the 8 × 8 chess grid based on the grayscale information of the input image. On the other hand, we exploit computer graphics techniques for the 3D modeling and rendering of the pieces together with the chessboard. Using 2D–3D correspondences, we are able to recover the 3D camera poses and determine game state transitions during the gaming process. Experimental study based on both simulated and real–world scenarios demonstrates the feasibility and effectiveness of our approach.