GAURA: Generalizable Approach for Unified Restoration and Rendering of Arbitrary Views
This addresses the challenge of robust 3D scene reconstruction from degraded images for applications in computer vision and graphics, offering a flexible solution that can adapt to new degradations with minimal data.
The paper tackles the problem of neural rendering failing with imperfect input images like low-light or hazy conditions, proposing GAURA, a generalizable method that achieves high-fidelity novel view synthesis under multiple degradations and outperforms state-of-the-art on benchmarks such as low-light enhancement and dehazing.
Neural rendering methods can achieve near-photorealistic image synthesis of scenes from posed input images. However, when the images are imperfect, e.g., captured in very low-light conditions, state-of-the-art methods fail to reconstruct high-quality 3D scenes. Recent approaches have tried to address this limitation by modeling various degradation processes in the image formation model; however, this limits them to specific image degradations. In this paper, we propose a generalizable neural rendering method that can perform high-fidelity novel view synthesis under several degradations. Our method, GAURA, is learning-based and does not require any test-time scene-specific optimization. It is trained on a synthetic dataset that includes several degradation types. GAURA outperforms state-of-the-art methods on several benchmarks for low-light enhancement, dehazing, deraining, and on-par for motion deblurring. Further, our model can be efficiently fine-tuned to any new incoming degradation using minimal data. We thus demonstrate adaptation results on two unseen degradations, desnowing and removing defocus blur. Code and video results are available at vinayak-vg.github.io/GAURA.