SonarSweep: Fusing Sonar and Vision for Robust 3D Reconstruction via Plane Sweeping
This addresses the challenge of robust 3D reconstruction for underwater robotics and exploration, offering a novel fusion approach to overcome limitations of single-modality methods.
The paper tackled the problem of 3D reconstruction in visually-degraded underwater environments by introducing SonarSweep, a deep learning framework that fuses sonar and vision data, resulting in dense and accurate depth maps that significantly outperform state-of-the-art methods, especially in high turbidity.
Accurate 3D reconstruction in visually-degraded underwater environments remains a formidable challenge. Single-modality approaches are insufficient: vision-based methods fail due to poor visibility and geometric constraints, while sonar is crippled by inherent elevation ambiguity and low resolution. Consequently, prior fusion technique relies on heuristics and flawed geometric assumptions, leading to significant artifacts and an inability to model complex scenes. In this paper, we introduce SonarSweep, a novel, end-to-end deep learning framework that overcomes these limitations by adapting the principled plane sweep algorithm for cross-modal fusion between sonar and visual data. Extensive experiments in both high-fidelity simulation and real-world environments demonstrate that SonarSweep consistently generates dense and accurate depth maps, significantly outperforming state-of-the-art methods across challenging conditions, particularly in high turbidity. To foster further research, we will publicly release our code and a novel dataset featuring synchronized stereo-camera and sonar data, the first of its kind.