UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Rendering
This addresses the need for geometrically consistent and efficient multimodal 3D scene reconstruction for applications in computer vision and graphics.
The paper tackles the problem of high-fidelity multimodal 3D reconstruction by proposing UniGS, a unified framework based on 3D Gaussian Splatting that simultaneously renders RGB images, depth maps, surface normals, and semantic logits. The result is state-of-the-art reconstruction accuracy across all modalities, validated through quantitative and qualitative experiments.
In this paper, we propose UniGS, a unified map representation and differentiable framework for high-fidelity multimodal 3D reconstruction based on 3D Gaussian Splatting. Our framework integrates a CUDA-accelerated rasterization pipeline capable of rendering photo-realistic RGB images, geometrically accurate depth maps, consistent surface normals, and semantic logits simultaneously. We redesign the rasterization to render depth via differentiable ray-ellipsoid intersection rather than using Gaussian centers, enabling effective optimization of rotation and scale attribute through analytic depth gradients. Furthermore, we derive the analytic gradient formulation for surface normal rendering, ensuring geometric consistency among reconstructed 3D scenes. To improve computational and storage efficiency, we introduce a learnable attribute that enables differentiable pruning of Gaussians with minimal contribution during training. Quantitative and qualitative experiments demonstrate state-of-the-art reconstruction accuracy across all modalities, validating the efficacy of our geometry-aware paradigm. Source code and multimodal viewer will be available on GitHub.