Single-image Full-body Human Relighting
This work addresses the challenge of realistic human relighting in images for applications in computer graphics and vision, representing an incremental improvement over prior methods.
The paper tackles the problem of relighting full-body humans from a single image by introducing a data-driven method that models both diffuse and specular reflectance and includes a light-dependent residual term, achieving state-of-the-art performance on synthetic images and photographs.
We present a single-image data-driven method to automatically relight images with full-body humans in them. Our framework is based on a realistic scene decomposition leveraging precomputed radiance transfer (PRT) and spherical harmonics (SH) lighting. In contrast to previous work, we lift the assumptions on Lambertian materials and explicitly model diffuse and specular reflectance in our data. Moreover, we introduce an additional light-dependent residual term that accounts for errors in the PRT-based image reconstruction. We propose a new deep learning architecture, tailored to the decomposition performed in PRT, that is trained using a combination of L1, logarithmic, and rendering losses. Our model outperforms the state of the art for full-body human relighting both with synthetic images and photographs.