All-Weather Deep Outdoor Lighting Estimation
This addresses the challenge of outdoor lighting estimation for computer vision applications, enabling more realistic rendering in varying weather, but it is incremental as it builds on existing models and datasets.
The paper tackles the problem of predicting HDR outdoor illumination from a single LDR image under any weather condition, achieving state-of-the-art performance in both panorama and single image networks.
We present a neural network that predicts HDR outdoor illumination from a single LDR image. At the heart of our work is a method to accurately learn HDR lighting from LDR panoramas under any weather condition. We achieve this by training another CNN (on a combination of synthetic and real images) to take as input an LDR panorama, and regress the parameters of the Lalonde-Matthews outdoor illumination model. This model is trained such that it a) reconstructs the appearance of the sky, and b) renders the appearance of objects lit by this illumination. We use this network to label a large-scale dataset of LDR panoramas with lighting parameters and use them to train our single image outdoor lighting estimation network. We demonstrate, via extensive experiments, that both our panorama and single image networks outperform the state of the art, and unlike prior work, are able to handle weather conditions ranging from fully sunny to overcast skies.