Leveraging Sparse LiDAR for RAFT-Stereo: A Depth Pre-Fill Perspective
This addresses a domain-specific problem for autonomous driving or robotics where sparse LiDAR data is common, offering an incremental improvement over prior LiDAR-guided stereo methods.
The paper tackled the problem of stereo matching accuracy degrading with sparse LiDAR data by proposing a pre-filling method to inject LiDAR depth into the RAFT-Stereo framework, resulting in GRAFT-Stereo significantly outperforming existing methods under sparse conditions across datasets.
We investigate LiDAR guidance within the RAFT-Stereo framework, aiming to improve stereo matching accuracy by injecting precise LiDAR depth into the initial disparity map. We find that the effectiveness of LiDAR guidance drastically degrades when the LiDAR points become sparse (e.g., a few hundred points per frame), and we offer a novel explanation from a signal processing perspective. This insight leads to a surprisingly simple solution that enables LiDAR-guided RAFT-Stereo to thrive: pre-filling the sparse initial disparity map with interpolation. Interestingly, we find that pre-filling is also effective when injecting LiDAR depth into image features via early fusion, but for a fundamentally different reason, necessitating a distinct pre-filling approach. By combining both solutions, the proposed Guided RAFT-Stereo (GRAFT-Stereo) significantly outperforms existing LiDAR-guided methods under sparse LiDAR conditions across various datasets. We hope this study inspires more effective LiDAR-guided stereo methods.