Visual Descriptor Learning from Monocular Video
This addresses the problem of reducing reliance on expensive labeled data or RGB-D videos for correspondence estimation in computer vision applications like tracking and mapping, though it is incremental as it builds on existing deep learning methods.
The paper tackles dense correspondence estimation in RGB images by learning visual descriptors from monocular video using a fully convolutional network trained with contrastive loss and optical flow, achieving good performance on test data with the same background and generalization to new backgrounds.
Correspondence estimation is one of the most widely researched and yet only partially solved area of computer vision with many applications in tracking, mapping, recognition of objects and environment. In this paper, we propose a novel way to estimate dense correspondence on an RGB image where visual descriptors are learned from video examples by training a fully convolutional network. Most deep learning methods solve this by training the network with a large set of expensive labeled data or perform labeling through strong 3D generative models using RGB-D videos. Our method learns from RGB videos using contrastive loss, where relative labeling is estimated from optical flow. We demonstrate the functionality in a quantitative analysis on rendered videos, where ground truth information is available. Not only does the method perform well on test data with the same background, it also generalizes to situations with a new background. The descriptors learned are unique and the representations determined by the network are global. We further show the applicability of the method to real-world videos.