Real-Time Monocular 4D Face Reconstruction using the LSFM models
This enables interactive facial modeling for applications like gaming or virtual reality, but it is incremental as it applies existing LSFM models to real-time use.
The authors tackled real-time 4D face reconstruction from a single webcam by implementing a system that runs interactively on a commodity laptop, accurately reconstructing and visualizing facial identity, pose, and expressions in real-time.
4D face reconstruction from a single camera is a challenging task, especially when it is required to be performed in real time. We demonstrate a system of our own implementation that solves this task accurately and runs in real time on a commodity laptop, using a webcam as the only input. Our system is interactive, allowing the user to freely move their head and show various expressions while standing in front of the camera. As a result, the put forward system both reconstructs and visualises the identity of the subject in the correct pose along with the acted facial expressions in real-time. The 4D reconstruction in our framework is based on the recently-released Large-Scale Facial Models (LSFM) \cite{LSFM1, LSFM2}, which are the largest-scale 3D Morphable Models of facial shapes ever constructed, based on a dataset of more than 10,000 facial identities from a wide range of gender, age and ethnicity combinations. This is the first real-time demo that gives users the opportunity to test in practice the capabilities of the recently-released Large-Scale Facial Models (LSFM)