LumiGauss: Relightable Gaussian Splatting in the Wild
This addresses the problem of creating complex 3D assets efficiently for users in fields like gaming and visualization, though it appears incremental as it builds on existing Gaussian Splatting methods.
The paper tackles the challenge of decoupling lighting from geometry in 3D reconstruction from unconstrained photo collections, introducing LumiGauss, which achieves high-quality scene reconstructions and enables realistic lighting synthesis under novel environment maps, with superior performance on the NeRF-OSR dataset.
Decoupling lighting from geometry using unconstrained photo collections is notoriously challenging. Solving it would benefit many users as creating complex 3D assets takes days of manual labor. Many previous works have attempted to address this issue, often at the expense of output fidelity, which questions the practicality of such methods. We introduce LumiGauss - a technique that tackles 3D reconstruction of scenes and environmental lighting through 2D Gaussian Splatting. Our approach yields high-quality scene reconstructions and enables realistic lighting synthesis under novel environment maps. We also propose a method for enhancing the quality of shadows, common in outdoor scenes, by exploiting spherical harmonics properties. Our approach facilitates seamless integration with game engines and enables the use of fast precomputed radiance transfer. We validate our method on the NeRF-OSR dataset, demonstrating superior performance over baseline methods. Moreover, LumiGauss can synthesize realistic images for unseen environment maps. Our code: https://github.com/joaxkal/lumigauss.