Face Reflectance and Geometry Modeling via Differentiable Ray Tracing
This work addresses the challenge of detailed 3D face modeling for applications in computer vision and graphics, representing an incremental improvement with novel method integration.
The paper tackled the problem of automatically reconstructing 3D faces from monocular images by disentangling geometry, reflectance, and self-shadows, using differentiable ray tracing to achieve robust reconstruction and explicit control over parameters like expressions and lighting.
We present a novel strategy to automatically reconstruct 3D faces from monocular images with explicitly disentangled facial geometry (pose, identity and expression), reflectance (diffuse and specular albedo), and self-shadows. The scene lights are modeled as a virtual light stage with pre-oriented area lights used in conjunction with differentiable Monte-Carlo ray tracing to optimize the scene and face parameters. With correctly disentangled self-shadows and specular reflection parameters, we can not only obtain robust facial geometry reconstruction, but also gain explicit control over these parameters, with several practical applications. We can change facial expressions with accurate resultant self-shadows or relight the scene and obtain accurate specular reflection and several other parameter combinations.