IM360: Large-scale Indoor Mapping with 360 Cameras
This addresses the challenge of accurate 3D mapping in large indoor environments with occlusions and textureless regions, representing a strong domain-specific advancement.
The paper tackles the problem of large-scale indoor 3D mapping by proposing IM360, a pipeline that uses 360 cameras and integrates a spherical camera model into Structure-from-Motion, resulting in a 3.5 PSNR increase in textured mesh reconstruction and state-of-the-art performance on datasets like Matterport3D and Stanford2D3D.
We present a novel 3D mapping pipeline for large-scale indoor environments. To address the significant challenges in large-scale indoor scenes, such as prevalent occlusions and textureless regions, we propose IM360, a novel approach that leverages the wide field of view of omnidirectional images and integrates the spherical camera model into the Structure-from-Motion (SfM) pipeline. Our SfM utilizes dense matching features specifically designed for 360 images, demonstrating superior capability in image registration. Furthermore, with the aid of mesh-based neural rendering techniques, we introduce a texture optimization method that refines texture maps and accurately captures view-dependent properties by combining diffuse and specular components. We evaluate our pipeline on large-scale indoor scenes, demonstrating its effectiveness in real-world scenarios. In practice, IM360 demonstrates superior performance, achieving a 3.5 PSNR increase in textured mesh reconstruction. We attain state-of-the-art performance in terms of camera localization and registration on Matterport3D and Stanford2D3D.