Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations
This work addresses the problem of inconsistent learning between contrastive objectives and data augmentations for researchers in self-supervised visual representation learning.
This paper introduces heterogeneous contrastive learning (HCL) to incorporate spatial information during the encoding stage of self-supervised visual representation learning. HCL achieves higher accuracy in instance discrimination and outperforms existing pre-training methods on downstream tasks while reducing pre-training costs by 50%.
Contrastive learning has achieved great success in self-supervised visual representation learning, but existing approaches mostly ignored spatial information which is often crucial for visual representation. This paper presents heterogeneous contrastive learning (HCL), an effective approach that adds spatial information to the encoding stage to alleviate the learning inconsistency between the contrastive objective and strong data augmentation operations. We demonstrate the effectiveness of HCL by showing that (i) it achieves higher accuracy in instance discrimination and (ii) it surpasses existing pre-training methods in a series of downstream tasks while shrinking the pre-training costs by half. More importantly, we show that our approach achieves higher efficiency in visual representations, and thus delivers a key message to inspire the future research of self-supervised visual representation learning.