Augmented World: Real Time Gesture Based Image Processing Tool with Intel Real Sense™ Technology

Abstract

Intel RealSense™ is an exciting new technology that offers innovative multimodal human computer interaction with hand tracking, face tracking, emotion detection, speech synthesis and voice recognition. The technology supports real time background separation by combining the depth and RGB streams of RealSense™ camera. Several past works have provided robust techniques for background separation. Background separated frame is used to augment segmented user photo in an artificial scene. In this work we propose a novel real time technique for augmenting the user in an artificial scene and apply various image filters on the constructed scene. We also propose a “Split Grid” based novel hand gesture driven UI control system that enables the user an intuitive way of working with the system with on-air hand movement and gesture. The system can be used in a wide range of applications like puppetry, photo retouching, and video recording on augmented scene .Usability tests performed by both amateur as well as professional digital photo artists proves ease and efficiency of the tool in generating quick wonderful photo effect in multimodal environment.