GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction
It addresses the problem of efficiently selecting informative viewpoints for active 3D reconstruction, which is incremental as it builds on existing methods like 3D Gaussian Splatting.
This paper tackles the challenge of real-time active view selection for 3D reconstruction by introducing GauSS-MI, a criterion based on Shannon Mutual Information to quantify visual uncertainty in Gaussian splatting, resulting in superior visual quality and reconstruction efficiency in experiments.
This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction. Visual quality is a critical aspect of 3D reconstruction. Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models. Nonetheless, the efficient and effective acquisition of input images for reconstruction-specifically, the selection of the most informative viewpoint-remains an open challenge, which is crucial for active reconstruction. Existing studies have primarily focused on evaluating geometric completeness and exploring unobserved or unknown regions, without direct evaluation of the visual uncertainty within the reconstruction model. To address this gap, this paper introduces a probabilistic model that quantifies visual uncertainty for each Gaussian. Leveraging Shannon Mutual Information, we formulate a criterion, Gaussian Splatting Shannon Mutual Information (GauSS-MI), for real-time assessment of visual mutual information from novel viewpoints, facilitating the selection of next best view. GauSS-MI is implemented within an active reconstruction system integrated with a view and motion planner. Extensive experiments across various simulated and real-world scenes showcase the superior visual quality and reconstruction efficiency performance of the proposed system.