Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence
This work addresses the challenge of limited ground truth data for scene understanding tasks, offering a joint framework that could benefit applications in robotics and autonomous driving, though it is incremental in combining existing tasks.
The paper tackled the problem of jointly learning stereo matching and optical flow estimation by proposing a single network that uses a geometric connection as an unsupervised signal, achieving favorable performance against state-of-the-art baselines on the KITTI benchmark dataset.
Stereo matching and flow estimation are two essential tasks for scene understanding, spatially in 3D and temporally in motion. Existing approaches have been focused on the unsupervised setting due to the limited resource to obtain the large-scale ground truth data. To construct a self-learnable objective, co-related tasks are often linked together to form a joint framework. However, the prior work usually utilizes independent networks for each task, thus not allowing to learn shared feature representations across models. In this paper, we propose a single and principled network to jointly learn spatiotemporal correspondence for stereo matching and flow estimation, with a newly designed geometric connection as the unsupervised signal for temporally adjacent stereo pairs. We show that our method performs favorably against several state-of-the-art baselines for both unsupervised depth and flow estimation on the KITTI benchmark dataset.