PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive Representation Learning
This work addresses a specific optimization issue in learning with noisy labels, offering an incremental improvement for researchers in robust machine learning.
The paper tackles the problem of performance degradation when combining contrastive representation learning with learning from noisy labels in an end-to-end framework, proposing PLReMix with a pseudo-label relaxed contrastive loss that filters inappropriate negative pairs and integrates with other methods to improve performance on benchmark datasets.
Recently, the usage of Contrastive Representation Learning (CRL) as a pre-training technique improves the performance of learning with noisy labels (LNL) methods. However, instead of pre-training, when trivially combining CRL loss with LNL methods as an end-to-end framework, the empirical experiments show severe degeneration of the performance. We verify through experiments that this issue is caused by optimization conflicts of losses and propose an end-to-end \textbf{PLReMix} framework by introducing a Pseudo-Label Relaxed (PLR) contrastive loss. This PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs, alleviating the loss conflicts by trivially combining these losses. The proposed PLR loss is pluggable and we have integrated it into other LNL methods, observing their improved performance. Furthermore, a two-dimensional Gaussian Mixture Model is adopted to distinguish clean and noisy samples by leveraging semantic information and model outputs simultaneously. Experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Code is available at \url{https://github.com/lxysl/PLReMix}.