Coarse Is Better? A New Pipeline Towards Self-Supervised Learning with Uncurated Images
This addresses the challenge of applying self-supervised learning to uncurated, scene-based images, which is an incremental improvement over existing methods that rely on curated datasets.
The paper tackles the problem of self-supervised learning on uncurated images by proposing a pipeline that crops coarse object regions to create pseudo object-centric images, enabling standard SSL methods to be applied directly, and it outperforms existing methods like MoCo-v2, DenseCL, and MAE on classification, detection, and segmentation tasks.
Most self-supervised learning (SSL) methods often work on curated datasets where the object-centric assumption holds. This assumption breaks down in uncurated images. Existing scene image SSL methods try to find the two views from original scene images that are well matched or dense, which is both complex and computationally heavy. This paper proposes a conceptually different pipeline: first find regions that are coarse objects (with adequate objectness), crop them out as pseudo object-centric images, then any SSL method can be directly applied as in a real object-centric dataset. That is, coarse crops benefits scene images SSL. A novel cropping strategy that produces coarse object box is proposed. The new pipeline and cropping strategy successfully learn quality features from uncurated datasets without ImageNet. Experiments show that our pipeline outperforms existing SSL methods (MoCo-v2, DenseCL and MAE) on classification, detection and segmentation tasks. We further conduct extensively ablations to verify that: 1) the pipeline do not rely on pretrained models; 2) the cropping strategy is better than existing object discovery methods; 3) our method is not sensitive to hyperparameters and data augmentations.