Self-Supervised Stereo Matching with Multi-Baseline Contrastive Learning
This addresses a key limitation in stereo vision for applications like robotics and autonomous driving, though it is an incremental improvement over existing self-supervised methods.
The paper tackles the problem of self-supervised stereo matching failing in occluded regions by proposing BaCon-Stereo, a contrastive learning framework that uses multi-baseline inputs and a teacher-student paradigm, achieving state-of-the-art results on KITTI 2015 and 2012 benchmarks.
Current self-supervised stereo matching relies on the photometric consistency assumption, which breaks down in occluded regions due to ill-posed correspondences. To address this issue, we propose BaCon-Stereo, a simple yet effective contrastive learning framework for self-supervised stereo network training in both non-occluded and occluded regions. We adopt a teacher-student paradigm with multi-baseline inputs, in which the stereo pairs fed into the teacher and student share the same reference view but differ in target views. Geometrically, regions occluded in the student's target view are often visible in the teacher's, making it easier for the teacher to predict in these regions. The teacher's prediction is rescaled to match the student's baseline and then used to supervise the student. We also introduce an occlusion-aware attention map to better guide the student in learning occlusion completion. To support training, we synthesize a multi-baseline dataset BaCon-20k. Extensive experiments demonstrate that BaCon-Stereo improves prediction in both occluded and non-occluded regions, achieves strong generalization and robustness, and outperforms state-of-the-art self-supervised methods on both KITTI 2015 and 2012 benchmarks. Our code and dataset will be released upon paper acceptance.