RNG: Relightable Neural Gaussians
This work addresses the challenging problem of creating relightable 3D assets for computer graphics and virtual reality applications, particularly for objects with complex or ill-defined shapes, by offering a faster and higher-quality solution.
This paper introduces Relightable Neural Gaussians (RNG), a 3D Gaussian Splatting-based framework for relighting 3D objects, including those with complex shapes like fur or fabric, without making assumptions about the shading model. The method achieves significantly faster training (1.3 hours) and rendering (60 frames per second) compared to a prior neural radiance field method, and produces higher-quality shadows than a concurrent 3DGS-based approach.
3D Gaussian Splatting (3DGS) has shown impressive results for the novel view synthesis task, where lighting is assumed to be fixed. However, creating relightable 3D assets, especially for objects with ill-defined shapes (fur, fabric, etc.), remains a challenging task. The decomposition between light, geometry, and material is ambiguous, especially if either smooth surface assumptions or surfacebased analytical shading models do not apply. We propose Relightable Neural Gaussians (RNG), a novel 3DGS-based framework that enables the relighting of objects with both hard surfaces or soft boundaries, while avoiding assumptions on the shading model. We condition the radiance at each point on both view and light directions. We also introduce a shadow cue, as well as a depth refinement network to improve shadow accuracy. Finally, we propose a hybrid forward-deferred fitting strategy to balance geometry and appearance quality. Our method achieves significantly faster training (1.3 hours) and rendering (60 frames per second) compared to a prior method based on neural radiance fields and produces higher-quality shadows than a concurrent 3DGS-based method. Project page: https://www.whois-jiahui.fun/project_pages/RNG.