3D Face Tracking and Texture Fusion in the Wild
This work addresses the problem of capturing facial expressions in uncontrolled environments for applications like animation or surveillance, but it is incremental as it builds on existing 3D Morphable Face Models and tracking methods.
The paper tackles real-time 3D face reconstruction from monocular videos in the wild, achieving robust performance on the 300-VW dataset without requiring person-specific training.
We present a fully automatic approach to real-time 3D face reconstruction from monocular in-the-wild videos. With the use of a cascaded-regressor based face tracking and a 3D Morphable Face Model shape fitting, we obtain a semi-dense 3D face shape. We further use the texture information from multiple frames to build a holistic 3D face representation from the video frames. Our system is able to capture facial expressions and does not require any person-specific training. We demonstrate the robustness of our approach on the challenging 300 Videos in the Wild (300-VW) dataset. Our real-time fitting framework is available as an open source library at http://4dface.org.