OmniGS: Fast Radiance Field Reconstruction using Omnidirectional Gaussian Splatting
This enables photorealistic 3D reconstruction from omnidirectional images, which is useful for domains like virtual reality and robotics, though it is an incremental extension of existing Gaussian splatting methods.
The paper tackles the limitation of 3D Gaussian Splatting to perspective images by introducing OmniGS, a system for fast radiance field reconstruction using omnidirectional images, achieving state-of-the-art quality and high rendering speed in experiments.
Photorealistic reconstruction relying on 3D Gaussian Splatting has shown promising potential in various domains. However, the current 3D Gaussian Splatting system only supports radiance field reconstruction using undistorted perspective images. In this paper, we present OmniGS, a novel omnidirectional Gaussian splatting system, to take advantage of omnidirectional images for fast radiance field reconstruction. Specifically, we conduct a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting. According to the derivatives, we then implement a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. We realize differentiable optimization of the omnidirectional radiance field without the requirement of cube-map rectification or tangent-plane approximation. Extensive experiments conducted in egocentric and roaming scenarios demonstrate that our method achieves state-of-the-art reconstruction quality and high rendering speed using omnidirectional images. The code will be publicly available.