Perceptually enhanced shadows for OST AR

Abstract

Shadows are important spatial cues for virtual objects in augmented reality (AR). The common assumption is that a shadow should be darker than its surroundings as it is the result of something obscuring the light source. However, darkening pixels is difficult in contemporary optical see-through (OST) head-mounted displays (HMDs) due to the additive display technologies that they employ. To address this issue, some previous methods create the illusion of darker shadow regions by brightening the surrounding pixels around a shadow, but the visibility of shadows created by such methods is limited in well-illuminated scenes and on surfaces with complex features. This paper presents a method that imbues cast shadows with specific hues to improve the perceptual contrast and, thus, the visibility of shadows in OST AR displays. The color of the cast shadow is computed as a complement of the perceptually weighted aggregate of the colors captured in the surrounding real scene. This approach is implemented on the Microsoft Hololens OST AR HMD, and coupled with a method for adjusting colors to compensate for the tint of the visor. The improvement in visibility is demonstrated with various sample scenes.