Estimating illumination distribution to generate realistic shadows in augmented reality

Abstract

Mobile devices are becoming powerful enough to realize augmented reality (AR) application. This paper introduces two AR methods to estimate an environmental illumination distribution of a scene. In the first method, we extract the lighting direction and intensity from input images captured with a front-side camera of a mobile device, using its orientation sensor. The second method extracts shadow regions cast by three dimensional (3D) AR marker of known shape and size. Because previous methods examine per pixel shadow intensity, their performances are much affected by the number of sampling points, positions, and threshold values. By using a simple binary operation between the previously clustered shadow regions and the threshold real shadow regions, we can compute efficiently their relative area proportions according to threshold values. This area-based method can overcome point sampling problem and threshold value selection. Experiment results demonstrate that the proposed methods generate natural image with multiple smooth shadows in real-time.