UniVIP: A Unified Framework for Self-Supervised Visual Pre-training
This addresses the need for more versatile self-supervised visual pre-training that works across different image types, offering incremental improvements over existing methods.
The paper tackles the problem that self-supervised learning methods are limited to single-centric-object images by proposing UniVIP, a unified framework that learns visual representations from both single-centric-object and non-iconic datasets, achieving state-of-the-art transfer performance on tasks like image classification and object detection, with improvements such as outperforming BYOL by 2.5% in linear probing on ImageNet.
Self-supervised learning (SSL) holds promise in leveraging large amounts of unlabeled data. However, the success of popular SSL methods has limited on single-centric-object images like those in ImageNet and ignores the correlation among the scene and instances, as well as the semantic difference of instances in the scene. To address the above problems, we propose a Unified Self-supervised Visual Pre-training (UniVIP), a novel self-supervised framework to learn versatile visual representations on either single-centric-object or non-iconic dataset. The framework takes into account the representation learning at three levels: 1) the similarity of scene-scene, 2) the correlation of scene-instance, 3) the discrimination of instance-instance. During the learning, we adopt the optimal transport algorithm to automatically measure the discrimination of instances. Massive experiments show that UniVIP pre-trained on non-iconic COCO achieves state-of-the-art transfer performance on a variety of downstream tasks, such as image classification, semi-supervised learning, object detection and segmentation. Furthermore, our method can also exploit single-centric-object dataset such as ImageNet and outperforms BYOL by 2.5% with the same pre-training epochs in linear probing, and surpass current self-supervised object detection methods on COCO dataset, demonstrating its universality and potential.