Deep Probabilistic Feature-metric Tracking
This addresses dense image alignment for real-world applications like robotics or AR/VR, though it appears incremental as it builds on existing feature-metric and optimization techniques.
The paper tackles dense image alignment from RGB-D images under challenging conditions by proposing a framework that learns pixel-wise deep feature maps and uncertainty maps to formulate a probabilistic feature-metric residual optimized with Gauss-Newton. The method achieves state-of-the-art performance on the TUM RGB-D and 3D rigid object tracking datasets.
Dense image alignment from RGB-D images remains a critical issue for real-world applications, especially under challenging lighting conditions and in a wide baseline setting. In this paper, we propose a new framework to learn a pixel-wise deep feature map and a deep feature-metric uncertainty map predicted by a Convolutional Neural Network (CNN), which together formulate a deep probabilistic feature-metric residual of the two-view constraint that can be minimised using Gauss-Newton in a coarse-to-fine optimisation framework. Furthermore, our network predicts a deep initial pose for faster and more reliable convergence. The optimisation steps are differentiable and unrolled to train in an end-to-end fashion. Due to its probabilistic essence, our approach can easily couple with other residuals, where we show a combination with ICP. Experimental results demonstrate state-of-the-art performances on the TUM RGB-D dataset and the 3D rigid object tracking dataset. We further demonstrate our method's robustness and convergence qualitatively.