Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
This work addresses the issue of noisy labels for deep learning practitioners, offering an incremental improvement over existing global estimation and filtering approaches.
The paper tackles the problem of noisy labels harming deep learning models by proposing a method that selectively identifies and corrects noisy samples based on loss distribution, applying a noise transition matrix to correct losses for noisy data while preserving clean samples. It reports significant improvements in accuracy and robustness on standard image datasets and a biological dataset compared to traditional methods.
Falsely annotated samples, also known as noisy labels, can significantly harm the performance of deep learning models. Two main approaches for learning with noisy labels are global noise estimation and data filtering. Global noise estimation approximates the noise across the entire dataset using a noise transition matrix, but it can unnecessarily adjust correct labels, leaving room for local improvements. Data filtering, on the other hand, discards potentially noisy samples but risks losing valuable data. Our method identifies potentially noisy samples based on their loss distribution. We then apply a selection process to separate noisy and clean samples and learn a noise transition matrix to correct the loss for noisy samples while leaving the clean data unaffected, thereby improving the training process. Our approach ensures robust learning and enhanced model performance by preserving valuable information from noisy samples and refining the correction process. We applied our method to standard image datasets (MNIST, CIFAR-10, and CIFAR-100) and a biological scRNA-seq cell-type annotation dataset. We observed a significant improvement in model accuracy and robustness compared to traditional methods.