Multi-level Supervised Contrastive Learning
This addresses the issue of capturing complex similarities in representation learning for multi-label and hierarchical classification tasks, but it is incremental as it builds on existing contrastive learning frameworks.
The paper tackles the problem of standard contrastive learning ignoring multiple aspects of similarity between samples, which leads to suboptimal performance, especially with limited data, by proposing a multilevel supervised contrastive learning method that outperforms state-of-the-art methods in experiments on text and image datasets.
Contrastive learning is a well-established paradigm in representation learning. The standard framework of contrastive learning minimizes the distance between "similar" instances and maximizes the distance between dissimilar ones in the projection space, disregarding the various aspects of similarity that can exist between two samples. Current methods rely on a single projection head, which fails to capture the full complexity of different aspects of a sample, leading to suboptimal performance, especially in scenarios with limited training data. In this paper, we present a novel supervised contrastive learning method in a unified framework called multilevel contrastive learning (MLCL), that can be applied to both multi-label and hierarchical classification tasks. The key strength of the proposed method is the ability to capture similarities between samples across different labels and/or hierarchies using multiple projection heads. Extensive experiments on text and image datasets demonstrate that the proposed approach outperforms state-of-the-art contrastive learning methods