2D Image Relighting with Image-to-Image Translation
This addresses image manipulation for computer vision applications, but is incremental as it builds on existing GAN methods and datasets.
The paper tackled the ill-posed problem of 2D image relighting by changing light source direction using GANs, specifically pix2pix trained on the VIDIT dataset, resulting in 8 neural networks capable of converting any direction to one of 8 specific ones and a CNN for light direction identification.
With the advent of Generative Adversarial Networks (GANs), a finer level of control in manipulating various features of an image has become possible. One example of such fine manipulation is changing the position of the light source in a scene. This is fundamentally an ill-posed problem, since it requires understanding the scene geometry to generate proper lighting effects. This problem is not a trivial one and can become even more complicated if we want to change the direction of the light source from any direction to a specific one. Here we provide our attempt to solve this problem using GANs. Specifically, pix2pix [arXiv:1611.07004] trained with the dataset VIDIT [arXiv:2005.05460] which contains images of the same scene with different types of light temperature and 8 different light source positions (N, NE, E, SE, S, SW, W, NW). The results are 8 neural networks trained to be able to change the direction of the light source from any direction to one of the 8 previously mentioned. Additionally, we provide, as a tool, a simple CNN trained to identify the direction of the light source in an image.