An Augmentation Overlap Theory of Contrastive Learning
This work addresses a foundational problem in machine learning by clarifying the theoretical underpinnings of contrastive learning, which is incremental but provides new insights for researchers in the field.
The paper tackles the unclear working mechanism of self-supervised contrastive learning by proposing an augmentation overlap theory, which provides asymptotically closed bounds for downstream performance and leads to an unsupervised evaluation metric that aligns well with actual performance.
Recently, self-supervised contrastive learning has achieved great success on various tasks. However, its underlying working mechanism is yet unclear. In this paper, we first provide the tightest bounds based on the widely adopted assumption of conditional independence. Further, we relax the conditional independence assumption to a more practical assumption of augmentation overlap and derive the asymptotically closed bounds for the downstream performance. Our proposed augmentation overlap theory hinges on the insight that the support of different intra-class samples will become more overlapped under aggressive data augmentations, thus simply aligning the positive samples (augmented views of the same sample) could make contrastive learning cluster intra-class samples together. Moreover, from the newly derived augmentation overlap perspective, we develop an unsupervised metric for the representation evaluation of contrastive learning, which aligns well with the downstream performance almost without relying on additional modules. Code is available at https://github.com/PKU-ML/GARC.