Face Relighting with Geometrically Consistent Shadows
This work addresses a specific challenge in computer vision for applications like virtual reality or photo editing, but it is incremental as it builds on existing face relighting methods.
The paper tackles the problem of generating geometrically consistent hard shadows in face relighting, which previous methods struggled with, and demonstrates state-of-the-art performance on Multi-PIE and FFHQ datasets.
Most face relighting methods are able to handle diffuse shadows, but struggle to handle hard shadows, such as those cast by the nose. Methods that propose techniques for handling hard shadows often do not produce geometrically consistent shadows since they do not directly leverage the estimated face geometry while synthesizing them. We propose a novel differentiable algorithm for synthesizing hard shadows based on ray tracing, which we incorporate into training our face relighting model. Our proposed algorithm directly utilizes the estimated face geometry to synthesize geometrically consistent hard shadows. We demonstrate through quantitative and qualitative experiments on Multi-PIE and FFHQ that our method produces more geometrically consistent shadows than previous face relighting methods while also achieving state-of-the-art face relighting performance under directional lighting. In addition, we demonstrate that our differentiable hard shadow modeling improves the quality of the estimated face geometry over diffuse shading models.