Diminished reality for AR marker hiding based on image inpainting with reflection of luminance changes

Abstract

This paper proposes a new diminished reality technique which removes AR markers from a user's view image. In order to achieve natural marker hiding, three factors should be considered; (1) naturalness of texture generated on a marker area. (2) geometric consistency between consecutive frames, (3) photometric consistency between a marker area and its surrounding. In this study, assuming that an area around a marker is locally planar, the marker area in the first frame image is inpainted using the rectified image to achieve high-quality inpainting. The unique inpainted texture is overlaid on the marker region in subsequent frames according to camera pose for temporal geometric consistency. Global and local luminance changes around the marker are reflected to the inpainted texture for photometric consistency.