Learning Neural Exposure Fields for View Synthesis
This addresses a specific challenge in 3D reconstruction for real-world captures with exposure changes, offering a robust solution for applications like virtual reality or photography, but it is incremental as it builds on existing neural scene representations.
The paper tackles the problem of view synthesis in scenes with strong exposure variations, such as indoor-outdoor transitions, by introducing Neural Exposure Fields (NExF) to predict optimal exposure per 3D point, resulting in state-of-the-art performance with over 55% improvement on benchmarks and faster training.
Recent advances in neural scene representations have led to unprecedented quality in 3D reconstruction and view synthesis. Despite achieving high-quality results for common benchmarks with curated data, outputs often degrade for data that contain per image variations such as strong exposure changes, present, e.g., in most scenes with indoor and outdoor areas or rooms with windows. In this paper, we introduce Neural Exposure Fields (NExF), a novel technique for robustly reconstructing 3D scenes with high quality and 3D-consistent appearance from challenging real-world captures. In the core, we propose to learn a neural field predicting an optimal exposure value per 3D point, enabling us to optimize exposure along with the neural scene representation. While capture devices such as cameras select optimal exposure per image/pixel, we generalize this concept and perform optimization in 3D instead. This enables accurate view synthesis in high dynamic range scenarios, bypassing the need of post-processing steps or multi-exposure captures. Our contributions include a novel neural representation for exposure prediction, a system for joint optimization of the scene representation and the exposure field via a novel neural conditioning mechanism, and demonstrated superior performance on challenging real-world data. We find that our approach trains faster than prior works and produces state-of-the-art results on several benchmarks improving by over 55% over best-performing baselines.