3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction
This work addresses a specific bottleneck in 3D scene reconstruction for indoor environments, offering incremental improvements over existing methods.
The paper tackled the problem of reconstructing flat, texture-less surfaces in photometric scene reconstruction, which current methods handle poorly, by proposing a hybrid 2D/3D Gaussian representation that improves visual fidelity and geometric accuracy, achieving state-of-the-art depth estimation on ScanNet++ and ScanNetv2 datasets.
Recent advances in radiance fields and novel view synthesis enable creation of realistic digital twins from photographs. However, current methods struggle with flat, texture-less surfaces, creating uneven and semi-transparent reconstructions, due to an ill-conditioned photometric reconstruction objective. Surface reconstruction methods solve this issue but sacrifice visual quality. We propose a novel hybrid 2D/3D representation that jointly optimizes constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. Our end-to-end approach dynamically detects and refines planar regions, improving both visual fidelity and geometric accuracy. It achieves state-of-the-art depth estimation on ScanNet++ and ScanNetv2, and excels at mesh extraction without overfitting to a specific camera model, showing its effectiveness in producing high-quality reconstruction of indoor scenes.