Do Convnets Learn Correspondence?
This addresses the problem of understanding visual correspondence beyond object classification for researchers in computer vision, though it is incremental as it builds on existing convnet architectures.
The paper investigates whether convolutional neural networks (convnets) learn fine-grained correspondence for precise localization, finding that convnet features localize at a finer scale than their receptive fields and outperform hand-engineered features in keypoint prediction on PASCAL VOC 2011.
Convolutional neural nets (convnets) trained from massive labeled datasets have substantially improved the state-of-the-art in image classification and object detection. However, visual understanding requires establishing correspondence on a finer level than object category. Given their large pooling regions and training from whole-image labels, it is not clear that convnets derive their success from an accurate correspondence model which could be used for precise localization. In this paper, we study the effectiveness of convnet activation features for tasks requiring correspondence. We present evidence that convnet features localize at a much finer scale than their receptive field sizes, that they can be used to perform intraclass alignment as well as conventional hand-engineered features, and that they outperform conventional features in keypoint prediction on objects from PASCAL VOC 2011.