Multi-Similarity Contrastive Learning
This addresses generalization issues in contrastive learning for representation tasks like image classification, but it is incremental as it builds on existing contrastive methods.
The paper tackles the problem of contrastive learning methods failing to generalize due to ignoring multiple similarity relations, and proposes a multi-similarity contrastive loss that improves out-of-domain generalization, outperforming state-of-the-art baselines.
Given a similarity metric, contrastive methods learn a representation in which examples that are similar are pushed together and examples that are dissimilar are pulled apart. Contrastive learning techniques have been utilized extensively to learn representations for tasks ranging from image classification to caption generation. However, existing contrastive learning approaches can fail to generalize because they do not take into account the possibility of different similarity relations. In this paper, we propose a novel multi-similarity contrastive loss (MSCon), that learns generalizable embeddings by jointly utilizing supervision from multiple metrics of similarity. Our method automatically learns contrastive similarity weightings based on the uncertainty in the corresponding similarity, down-weighting uncertain tasks and leading to better out-of-domain generalization to new tasks. We show empirically that networks trained with MSCon outperform state-of-the-art baselines on in-domain and out-of-domain settings.