AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion
This work addresses underwater perception challenges for applications like construction and environmental monitoring, offering a novel multimodal fusion approach that is incremental in combining existing modalities.
The paper tackles the problem of 3D surface reconstruction in underwater environments with restricted motion baselines by developing AONeuS, a framework that fuses acoustic and optical sensor data, resulting in dramatic performance improvements over existing methods in simulations and experiments.
Underwater perception and 3D surface reconstruction are challenging problems with broad applications in construction, security, marine archaeology, and environmental monitoring. Treacherous operating conditions, fragile surroundings, and limited navigation control often dictate that submersibles restrict their range of motion and, thus, the baseline over which they can capture measurements. In the context of 3D scene reconstruction, it is well-known that smaller baselines make reconstruction more challenging. Our work develops a physics-based multimodal acoustic-optical neural surface reconstruction framework (AONeuS) capable of effectively integrating high-resolution RGB measurements with low-resolution depth-resolved imaging sonar measurements. By fusing these complementary modalities, our framework can reconstruct accurate high-resolution 3D surfaces from measurements captured over heavily-restricted baselines. Through extensive simulations and in-lab experiments, we demonstrate that AONeuS dramatically outperforms recent RGB-only and sonar-only inverse-differentiable-rendering--based surface reconstruction methods. A website visualizing the results of our paper is located at this address: https://aoneus.github.io/