Deep Outdoor Illumination Estimation
This addresses the challenge of realistic outdoor lighting estimation for applications like augmented reality, though it is incremental as it builds on existing CNN and panorama-based approaches.
The paper tackles the problem of estimating high-dynamic range outdoor illumination from a single low dynamic range image using a CNN trained on a large dataset of outdoor panoramas, resulting in a method that significantly outperforms previous solutions and enables photorealistic virtual object insertion.
We present a CNN-based technique to estimate high-dynamic range outdoor illumination from a single low dynamic range image. To train the CNN, we leverage a large dataset of outdoor panoramas. We fit a low-dimensional physically-based outdoor illumination model to the skies in these panoramas giving us a compact set of parameters (including sun position, atmospheric conditions, and camera parameters). We extract limited field-of-view images from the panoramas, and train a CNN with this large set of input image--output lighting parameter pairs. Given a test image, this network can be used to infer illumination parameters that can, in turn, be used to reconstruct an outdoor illumination environment map. We demonstrate that our approach allows the recovery of plausible illumination conditions and enables photorealistic virtual object insertion from a single image. An extensive evaluation on both the panorama dataset and captured HDR environment maps shows that our technique significantly outperforms previous solutions to this problem.