Deep Cost Ray Fusion for Sparse Depth Video Completion
This work addresses depth completion for robotics and autonomous systems, offering an incremental improvement in efficiency and performance.
The paper tackles sparse depth video completion by introducing RayFusion, a learning-based cost volume fusion framework that uses attention mechanisms on overlapped rays, achieving state-of-the-art or competitive results on datasets like KITTI, VOID, and ScanNetV2 with fewer parameters.
In this paper, we present a learning-based framework for sparse depth video completion. Given a sparse depth map and a color image at a certain viewpoint, our approach makes a cost volume that is constructed on depth hypothesis planes. To effectively fuse sequential cost volumes of the multiple viewpoints for improved depth completion, we introduce a learning-based cost volume fusion framework, namely RayFusion, that effectively leverages the attention mechanism for each pair of overlapped rays in adjacent cost volumes. As a result of leveraging feature statistics accumulated over time, our proposed framework consistently outperforms or rivals state-of-the-art approaches on diverse indoor and outdoor datasets, including the KITTI Depth Completion benchmark, VOID Depth Completion benchmark, and ScanNetV2 dataset, using much fewer network parameters.