SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition
This addresses the need for consistent lighting in images for applications like photography or computer vision, but it is incremental as it builds on existing relighting research with a focus on style transfer.
The paper tackles the problem of transferring lighting styles from a reference database to input images to achieve uniform lighting, presenting SILT, a self-supervised method that outperforms supervised relighting solutions on two datasets without requiring lighting supervision.
We present SILT, a Self-supervised Implicit Lighting Transfer method. Unlike previous research on scene relighting, we do not seek to apply arbitrary new lighting configurations to a given scene. Instead, we wish to transfer the lighting style from a database of other scenes, to provide a uniform lighting style regardless of the input. The solution operates as a two-branch network that first aims to map input images of any arbitrary lighting style to a unified domain, with extra guidance achieved through implicit image decomposition. We then remap this unified input domain using a discriminator that is presented with the generated outputs and the style reference, i.e. images of the desired illumination conditions. Our method is shown to outperform supervised relighting solutions across two different datasets without requiring lighting supervision.