InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction
This work addresses the challenge of ambiguity in sparse-view reconstruction for scene understanding, offering a plugin technique that enhances existing SDF-based neural surface representations, though it is incremental as it builds on prior implicit surface methods.
The paper tackles the problem of 3D surface reconstruction from sparse multi-view images in complex indoor scenes by proposing InfoNorm, a method that regularizes geometric modeling through mutual information shaping of surface normals to reduce dependence on pre-trained geometry estimation models. The result is improved surface reconstruction quality, as demonstrated in experiments with major state-of-the-art methods.
3D surface reconstruction from multi-view images is essential for scene understanding and interaction. However, complex indoor scenes pose challenges such as ambiguity due to limited observations. Recent implicit surface representations, such as Neural Radiance Fields (NeRFs) and signed distance functions (SDFs), employ various geometric priors to resolve the lack of observed information. Nevertheless, their performance heavily depends on the quality of the pre-trained geometry estimation models. To ease such dependence, we propose regularizing the geometric modeling by explicitly encouraging the mutual information among surface normals of highly correlated scene points. In this way, the geometry learning process is modulated by the second-order correlations from noisy (first-order) geometric priors, thus eliminating the bias due to poor generalization. Additionally, we introduce a simple yet effective scheme that utilizes semantic and geometric features to identify correlated points, enhancing their mutual information accordingly. The proposed technique can serve as a plugin for SDF-based neural surface representations. Our experiments demonstrate the effectiveness of the proposed in improving the surface reconstruction quality of major states of the arts. Our code is available at: \url{https://github.com/Muliphein/InfoNorm}.