R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild
This work addresses the challenge of relighting and reconstructing outdoor scenes with varying illumination for applications in computer vision and graphics, representing an incremental improvement over existing 3DGS methods.
The paper tackles the problem of 3D Gaussian Splatting (3DGS) being unsuitable for relighting tasks and struggling with outdoor scenes under changing lighting conditions, by introducing R3GW, which learns a relightable 3DGS representation for outdoor scenes, achieving state-of-the-art performance on the NeRF-OSR dataset and synthesizing photorealistic novel views under arbitrary illumination.
3D Gaussian Splatting (3DGS) has established itself as a leading technique for 3D reconstruction and novel view synthesis of static scenes, achieving outstanding rendering quality and fast training. However, the method does not explicitly model the scene illumination, making it unsuitable for relighting tasks. Furthermore, 3DGS struggles to reconstruct scenes captured in the wild by unconstrained photo collections featuring changing lighting conditions. In this paper, we present R3GW, a novel method that learns a relightable 3DGS representation of an outdoor scene captured in the wild. Our approach separates the scene into a relightable foreground and a non-reflective background (the sky), using two distinct sets of Gaussians. R3GW models view-dependent lighting effects in the foreground reflections by combining Physically Based Rendering with the 3DGS scene representation in a varying illumination setting. We evaluate our method quantitatively and qualitatively on the NeRF-OSR dataset, offering state-of-the-art performance and enhanced support for physically-based relighting of unconstrained scenes. Our method synthesizes photorealistic novel views under arbitrary illumination conditions. Additionally, our representation of the sky mitigates depth reconstruction artifacts, improving rendering quality at the sky-foreground boundary