Multi-Objective Interpolation Training for Robustness to Label Noise
This paper tackles the problem of label noise memorization in deep neural networks, which is a common issue for practitioners training models on real-world datasets.
The paper addresses the problem of deep neural networks memorizing noisy labels, which degrades performance. They propose Multi-Objective Interpolation Training (MOIT) that combines contrastive learning and classification, and a label noise detection method to treat noisy samples as unlabeled, achieving state-of-the-art results on synthetic and real-world noise benchmarks.
Deep neural networks trained with standard cross-entropy loss memorize noisy labels, which degrades their performance. Most research to mitigate this memorization proposes new robust classification loss functions. Conversely, we propose a Multi-Objective Interpolation Training (MOIT) approach that jointly exploits contrastive learning and classification to mutually help each other and boost performance against label noise. We show that standard supervised contrastive learning degrades in the presence of label noise and propose an interpolation training strategy to mitigate this behavior. We further propose a novel label noise detection method that exploits the robust feature representations learned via contrastive learning to estimate per-sample soft-labels whose disagreements with the original labels accurately identify noisy samples. This detection allows treating noisy samples as unlabeled and training a classifier in a semi-supervised manner to prevent noise memorization and improve representation learning. We further propose MOIT+, a refinement of MOIT by fine-tuning on detected clean samples. Hyperparameter and ablation studies verify the key components of our method. Experiments on synthetic and real-world noise benchmarks demonstrate that MOIT/MOIT+ achieves state-of-the-art results. Code is available at https://git.io/JI40X.