MaskBot: Real-time Robotic Projection Mapping with Head Motion Tracking

Abstract

The projection mapping systems on the human face are limited by the process latency and the users' movement. The area of the projection is restricted by the position of the projectors and cameras. We are introducing MaskBot, a real-time projection mapping system guided by a 6 Degrees of Freedom (DoF) collaborative robot. The collaborative robot locates the projector and camera in front of the user's face to increase the projection area and reduce the system's latency. A webcam is used to detect the face orientation and to measure the robot-user distance. Based on this information we modify the projection size and orientation. MaskBot projects different images on the user's face, such as face modifications, make-up, and logos. In contrast to the existing methods, the presented system is the first that introduces a robotic projection mapping. One of the prospective applications is to acquire a dataset of adversarial images to challenge face detection DNN systems, such as Face ID.