Interactive projection mapping

Abstract

We propose an interactive projection mapping that provides ultra-realistic effects by mixing realistic and diverse visual effects with the users interactions. To achieve high quality and real-time generation of the visual effects, we create pre-rendered movies with layered architecture including the target itself, its background, and additional effects. Based on the user 's interactions estimated with a depth camera, the movies are superimposed for various visual patterns in real time. These movies are projected on a real space by using common 3D projection mapping techniques, and it feels like the resultant projection is part of the real world. In this research, we implemented a wall breaking game by throwing a virtual and an actual ball at a real wall.