All-frequency Full-body Human Image Relighting
This work addresses the challenge of high-frequency shadow representation in portrait editing, offering an incremental improvement over existing neural network-based methods.
The paper tackles the problem of reproducing physically-based shadows and shading in human image relighting, proposing a two-stage method that approximates environment lighting with area light sources and achieves plausible all-frequency shadow and shading reproduction.
Relighting of human images enables post-photography editing of lighting effects in portraits. The current mainstream approach uses neural networks to approximate lighting effects without explicitly accounting for the principle of physical shading. As a result, it often has difficulty representing high-frequency shadows and shading. In this paper, we propose a two-stage relighting method that can reproduce physically-based shadows and shading from low to high frequencies. The key idea is to approximate an environment light source with a set of a fixed number of area light sources. The first stage employs supervised inverse rendering from a single image using neural networks and calculates physically-based shading. The second stage then calculates shadow for each area light and sums up to render the final image. We propose to make soft shadow mapping differentiable for the area-light approximation of environment lighting. We demonstrate that our method can plausibly reproduce all-frequency shadows and shading caused by environment illumination, which have been difficult to reproduce using existing methods.