SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing
This work addresses label noise robustness in deep learning, which is crucial for real-world applications where data is often imperfect, but it is incremental as it builds upon existing Mixup-based methods.
The paper tackled the problem of deep neural networks memorizing noisy labels by proposing SelectMix, a confidence-guided mixing framework that selectively blends uncertain samples with confident peers and uses soft labels, resulting in consistent outperformance of baseline methods on multiple synthetic and real-world datasets.
Deep neural networks tend to memorize noisy labels, severely degrading their generalization performance. Although Mixup has demonstrated effectiveness in improving generalization and robustness, existing Mixup-based methods typically perform indiscriminate mixing without principled guidance on sample selection and mixing strategy, inadvertently propagating noisy supervision. To overcome these limitations, we propose SelectMix, a confidence-guided mixing framework explicitly tailored for noisy labels. SelectMix first identifies potentially noisy or ambiguous samples through confidence based mismatch analysis using K-fold cross-validation, then selectively blends identified uncertain samples with confidently predicted peers from their potential classes. Furthermore, SelectMix employs soft labels derived from all classes involved in the mixing process, ensuring the labels accurately represent the composition of the mixed samples, thus aligning supervision signals closely with the actual mixed inputs. Through extensive theoretical analysis and empirical evaluations on multiple synthetic (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100) and real-world benchmark datasets (CIFAR-N, MNIST and Clothing1M), we demonstrate that SelectMix consistently outperforms strong baseline methods, validating its effectiveness and robustness in learning with noisy labels.