Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework
This work addresses the problem of limited efficacy in unseen data and downstream tasks for representation learning, offering a domain-specific solution for multi-label classification.
The paper tackles the limitation of single supervisory signals in contrastive learning by introducing a hierarchical multi-label framework that leverages all available labels and preserves class relationships, resulting in improved performance on various tasks compared to baseline supervised and self-supervised methods.
Current contrastive learning frameworks focus on leveraging a single supervisory signal to learn representations, which limits the efficacy on unseen data and downstream tasks. In this paper, we present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes. We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint. The loss function is data driven and automatically adapts to arbitrary multi-label structures. Experiments on several datasets show that our relationship-preserving embedding performs well on a variety of tasks and outperform the baseline supervised and self-supervised approaches. Code is available at https://github.com/salesforce/hierarchicalContrastiveLearning.