Friends and Foes in Learning from Noisy Labels
This work addresses the challenge of noisy labels in machine learning, which is critical for improving model robustness in real-world applications, but it is incremental as it builds on and refines existing methods.
The paper tackled the problem of learning from noisy labels by showing that commonly used CIFAR-based datasets and accuracy metrics are inappropriate, proposing new datasets and a valid evaluation metric. The resulting F&F method, which combines beneficial technical components like self-supervised learning and label correction, significantly outperforms existing methods on nCIFAR and Clothing1M datasets.
Learning from examples with noisy labels has attracted increasing attention recently. But, this paper will show that the commonly used CIFAR-based datasets and the accuracy evaluation metric used in the literature are both inappropriate in this context. An alternative valid evaluation metric and new datasets are proposed in this paper to promote proper research and evaluation in this area. Then, friends and foes are identified from existing methods as technical components that are either beneficial or detrimental to deep learning from noisy labeled examples, respectively, and this paper improves and combines technical components from the friends category, including self-supervised learning, new warmup strategy, instance filtering and label correction. The resulting F&F method significantly outperforms existing methods on the proposed nCIFAR datasets and the real-world Clothing1M dataset.