Learning Image Matching by Simply Watching Video
It addresses the computer vision problem of image matching for broader applicability, but is incremental as it builds on existing frame interpolation techniques.
The paper tackles image matching by training a CNN for frame interpolation using unsupervised learning from video sequences, then inverting it to obtain correspondences, achieving performance comparable to traditional methods.
This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching. We start from the insight that the problem of frame-interpolation implicitly solves for inter-frame correspondences. This permits the application of analysis-by-synthesis: we firstly train and apply a Convolutional Neural Network for frame-interpolation, then obtain correspondences by inverting the learned CNN. The key benefit behind this strategy is that the CNN for frame-interpolation can be trained in an unsupervised manner by exploiting the temporal coherency that is naturally contained in real-world video sequences. The present model therefore learns image matching by simply watching videos. Besides a promise to be more generally applicable, the presented approach achieves surprising performance comparable to traditional empirically designed methods.