Monocular Facial Appearance Capture in the Wild
This enables easier and cheaper high-fidelity facial capture for applications like animation or VR, though it is incremental as it builds on existing monocular capture methods.
The paper tackles the problem of reconstructing facial appearance properties from monocular video in unconstrained environments, achieving results that approach the fidelity of studio-based multi-view captures with a simpler and cheaper procedure.
We present a new method for reconstructing the appearance properties of human faces from a lightweight capture procedure in an unconstrained environment. Our method recovers the surface geometry, diffuse albedo, specular intensity and specular roughness from a monocular video containing a simple head rotation in-the-wild. Notably, we make no simplifying assumptions on the environment lighting, and we explicitly take visibility and occlusions into account. As a result, our method can produce facial appearance maps that approach the fidelity of studio-based multi-view captures, but with a far easier and cheaper procedure.