Designing An Illumination-Aware Network for Deep Image Relighting
This work addresses the challenge of creating satisfying lighting conditions in photography, which is laborious in reality, by developing a technology for image relighting, though it appears incremental as it builds on image-to-image translation and physical viewpoint ideas.
The paper tackles the problem of manipulating illumination in images as post-processing by proposing an Illumination-Aware Network (IAN) that progressively relights scenes from a single image, achieving better quantitative and qualitative results than previous state-of-the-art methods.
Lighting is a determining factor in photography that affects the style, expression of emotion, and even quality of images. Creating or finding satisfying lighting conditions, in reality, is laborious and time-consuming, so it is of great value to develop a technology to manipulate illumination in an image as post-processing. Although previous works have explored techniques based on the physical viewpoint for relighting images, extensive supervisions and prior knowledge are necessary to generate reasonable images, restricting the generalization ability of these works. In contrast, we take the viewpoint of image-to-image translation and implicitly merge ideas of the conventional physical viewpoint. In this paper, we present an Illumination-Aware Network (IAN) which follows the guidance from hierarchical sampling to progressively relight a scene from a single image with high efficiency. In addition, an Illumination-Aware Residual Block (IARB) is designed to approximate the physical rendering process and to extract precise descriptors of light sources for further manipulations. We also introduce a depth-guided geometry encoder for acquiring valuable geometry- and structure-related representations once the depth information is available. Experimental results show that our proposed method produces better quantitative and qualitative relighting results than previous state-of-the-art methods. The code and models are publicly available on https://github.com/NK-CS-ZZL/IAN.