An Asymmetric Contrastive Loss for Handling Imbalanced Datasets
This work addresses class imbalance in datasets, a common issue in machine learning, but is incremental as it modifies existing contrastive loss methods.
The authors tackled the problem of class imbalance in datasets by introducing an asymmetric contrastive loss (ACL) and its generalization, asymmetric focal contrastive loss (AFCL), which outperformed existing methods on FMNIST and ISIC 2018 datasets in terms of weighted and unweighted classification accuracies.
Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process is typically conducted using a two-stage training architecture, and it utilizes the contrastive loss (CL) for its feature learning. Contrastive learning has been shown to be quite successful in handling imbalanced datasets, in which some classes are overrepresented while some others are underrepresented. However, previous studies have not specifically modified CL for imbalanced datasets. In this work, we introduce an asymmetric version of CL, referred to as ACL, in order to directly address the problem of class imbalance. In addition, we propose the asymmetric focal contrastive loss (AFCL) as a further generalization of both ACL and focal contrastive loss (FCL). Results on the FMNIST and ISIC 2018 imbalanced datasets show that AFCL is capable of outperforming CL and FCL in terms of both weighted and unweighted classification accuracies. In the appendix, we provide a full axiomatic treatment on entropy, along with complete proofs.