MaskBot: Real-time Robotic Projection Mapping with Head Motion Tracking
This system addresses the problem of dynamic facial projection for users in interactive or entertainment settings, representing a novel robotic approach but with incremental technical improvements.
MaskBot tackles the limitations of latency and restricted projection area in facial projection mapping by using a 6-DoF collaborative robot to position the projector and camera normally to the user's face, achieving real-time operation and enabling applications like face modifications and adversarial dataset generation.
The projection mapping systems on the human face is limited by the latency and the movement of the users. The area of the projection is restricted by the position of the projectors and the cameras. We are introducing MaskBot, a real-time projection mapping system operated by a 6 Degrees of Freedom (DoF) collaborative robot. The collaborative robot locates the projector and camera in normal position to the face of the user to increase the projection area and to reduce the latency of the system. A webcam is used to detect the face and to sense the robot-user distance to modify the projection size and orientation. MaskBot projects different images on the face of the user, such as face modifications, make-up, and logos. In contrast to the existing methods, the presented system is the first that introduces a robotic projection mapping. One of the prospective applications is to acquire a dataset of adversarial images to challenge face detection DNN systems, such as Face ID.