Jo-SRC: A Contrastive Approach for Combating Noisy Labels
This addresses the issue of noisy labels in machine learning, which is a common problem in real-world datasets, but the approach appears incremental as it builds on existing sample selection strategies.
The paper tackles the problem of training deep neural networks with noisy labels, which degrades performance, by proposing Jo-SRC, a contrastive learning approach that jointly selects clean samples and applies consistency regularization, achieving superior results over state-of-the-art methods in experiments.
Due to the memorization effect in Deep Neural Networks (DNNs), training with noisy labels usually results in inferior model performance. Existing state-of-the-art methods primarily adopt a sample selection strategy, which selects small-loss samples for subsequent training. However, prior literature tends to perform sample selection within each mini-batch, neglecting the imbalance of noise ratios in different mini-batches. Moreover, valuable knowledge within high-loss samples is wasted. To this end, we propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency). Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution. Furthermore, we propose a joint loss to advance the model generalization performance by introducing consistency regularization. Extensive experiments have validated the superiority of our approach over existing state-of-the-art methods.