SfM on-the-fly: Get better 3D from What You Capture
This work addresses real-time 3D reconstruction for applications in photogrammetry, computer vision, and robotics, but it is incremental as it builds directly on a prior method.
The paper tackles the problem of improving real-time Structure from Motion (SfM) for 3D reconstruction by building on an existing method with three advancements: using HNSW graphs for faster image matching, a self-adaptive weighting strategy for robust bundle adjustment, and multiple agents for collaborative reconstruction, resulting in more complete and robust 3D reconstructions with high time efficiency.
In the last twenty years, Structure from Motion (SfM) has been a constant research hotspot in the fields of photogrammetry, computer vision, robotics etc., whereas real-time performance is just a recent topic of growing interest. This work builds upon the original on-the-fly SfM (Zhan et al., 2024) and presents an updated version with three new advancements to get better 3D from what you capture: (i) real-time image matching is further boosted by employing the Hierarchical Navigable Small World (HNSW) graphs, thus more true positive overlapping image candidates are faster identified; (ii) a self-adaptive weighting strategy is proposed for robust hierarchical local bundle adjustment to improve the SfM results; (iii) multiple agents are included for supporting collaborative SfM and seamlessly merge multiple 3D reconstructions into a complete 3D scene when commonly registered images appear. Various comprehensive experiments demonstrate that the proposed SfM method (named on-the-fly SfMv2) can generate more complete and robust 3D reconstructions in a high time-efficient way. Code is available at http://yifeiyu225.github.io/on-the-flySfMv2.github.io/.