Movement-induced Priors for Deep Stereo
This addresses the challenge of improving stereo depth estimation for handheld devices with low-quality sensors, though it is incremental as it builds on existing deep stereo methods.
The paper tackles the problem of stereo disparity estimation by fusing it with movement-induced prior information using a temporal Gaussian process with movement-driven kernels, leading to consistent improvements when combined with state-of-the-art deep stereo methods.
We propose a method for fusing stereo disparity estimation with movement-induced prior information. Instead of independent inference frame-by-frame, we formulate the problem as a non-parametric learning task in terms of a temporal Gaussian process prior with a movement-driven kernel for inter-frame reasoning. We present a hierarchy of three Gaussian process kernels depending on the availability of motion information, where our main focus is on a new gyroscope-driven kernel for handheld devices with low-quality MEMS sensors, thus also relaxing the requirement of having full 6D camera poses available. We show how our method can be combined with two state-of-the-art deep stereo methods. The method either work in a plug-and-play fashion with pre-trained deep stereo networks, or further improved by jointly training the kernels together with encoder-decoder architectures, leading to consistent improvement.