Augmented reality using optical flow

Abstract

The paper deals with the application of Lucas-Kanade optical flow algorithm to develop an augmented reality (AR) system. Merging of a live view of the physical real world with context-related computer-rendered images to create a mixed image is a challenging problem. A virtual object has to be located in the correct pose and position in real time and perspective. Besides the occlusion problem need to be taken into consideration. In the paper a computer-vision based method for AR systems based on the fiducial marker matching is proposed. For simplicity black square was used as the marker. This method consists of two main steps. The initial step uses Hough's transformation to detect the marker initial position and to select the marker tracked points. In the second step for each image frame these selected points are being tracked using Lucas-Kanade optical flow method. The positions of the selected points are used for calculating the pose and position of a virtual object. Unlike existing method proposed system using optical flow to increase speed performance. The examples of AR applications using the proposed algorithm are provided and discussed.