SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians
This addresses the challenge of efficiently reconstructing 3D scenes from unstructured photo collections for applications in computer vision and graphics, representing an incremental improvement over existing methods.
The paper tackles the problem of poor performance of 3D Gaussian Splatting on unstructured in-the-wild image collections by modeling appearance to handle photometric variations and training transient Gaussians for occluders, achieving state-of-the-art results with improved efficiency on diverse scenes.
Implicit neural representation methods have shown impressive advancements in learning 3D scenes from unstructured in-the-wild photo collections but are still limited by the large computational cost of volumetric rendering. More recently, 3D Gaussian Splatting emerged as a much faster alternative with superior rendering quality and training efficiency, especially for small-scale and object-centric scenarios. Nevertheless, this technique suffers from poor performance on unstructured in-the-wild data. To tackle this, we extend over 3D Gaussian Splatting to handle unstructured image collections. We achieve this by modeling appearance to seize photometric variations in the rendered images. Additionally, we introduce a new mechanism to train transient Gaussians to handle the presence of scene occluders in an unsupervised manner. Experiments on diverse photo collection scenes and multi-pass acquisition of outdoor landmarks show the effectiveness of our method over prior works achieving state-of-the-art results with improved efficiency.