TemporalStereo: Efficient Spatial-Temporal Stereo Matching Network
This addresses the problem of occlusions and reflective regions in stereo vision for applications like robotics and autonomous driving, with incremental improvements in efficiency and robustness.
The paper tackles stereo matching by introducing TemporalStereo, a network that efficiently uses spatio-temporal information from stereo sequences to improve accuracy, achieving state-of-the-art performance on multiple datasets.
We present TemporalStereo, a coarse-to-fine stereo matching network that is highly efficient, and able to effectively exploit the past geometry and context information to boost matching accuracy. Our network leverages sparse cost volume and proves to be effective when a single stereo pair is given. However, its peculiar ability to use spatio-temporal information across stereo sequences allows TemporalStereo to alleviate problems such as occlusions and reflective regions while enjoying high efficiency also in this latter case. Notably, our model -- trained once with stereo videos -- can run in both single-pair and temporal modes seamlessly. Experiments show that our network relying on camera motion is robust even to dynamic objects when running on videos. We validate TemporalStereo through extensive experiments on synthetic (SceneFlow, TartanAir) and real (KITTI 2012, KITTI 2015) datasets. Our model achieves state-of-the-art performance on any of these datasets. Code is available at \url{https://github.com/youmi-zym/TemporalStereo.git}.