Distilling Localization for Self-Supervised Representation Learning
This addresses a key bottleneck in self-supervised representation learning for computer vision tasks, offering incremental improvements over existing contrastive methods.
The paper tackled the problem of contrastive models being ineffective at localizing foreground objects, which limits discriminative feature extraction, by proposing a data-driven approach that estimates foreground saliency and creates augmentations to focus on foregrounds, achieving significant performance improvements in ImageNet classification and object detection on PASCAL VOC and MSCOCO.
Recent progress in contrastive learning has revolutionized unsupervised representation learning. Concretely, multiple views (augmentations) from the same image are encouraged to map to the similar embeddings, while views from different images are pulled apart. In this paper, through visualizing and diagnosing classification errors, we observe that current contrastive models are ineffective at localizing the foreground object, limiting their ability to extract discriminative high-level features. This is due to the fact that view generation process considers pixels in an image uniformly. To address this problem, we propose a data-driven approach for learning invariance to backgrounds. It first estimates foreground saliency in images and then creates augmentations by copy-and-pasting the foreground onto a variety of backgrounds. The learning still follows the instance discrimination pretext task, so that the representation is trained to disregard background content and focus on the foreground. We study a variety of saliency estimation methods, and find that most methods lead to improvements for contrastive learning. With this approach (DiLo), significant performance is achieved for self-supervised learning on ImageNet classification, and also for object detection on PASCAL VOC and MSCOCO.