Unsupervised Label Noise Modeling and Loss Correction
This addresses label noise in deep learning, which is a common issue in real-world datasets, offering a novel solution that outperforms recent state-of-the-art methods.
The paper tackles the problem of convolutional neural networks fitting mislabeled data by proposing an unsupervised method using a beta mixture model to estimate mislabeling probability and correct loss, combined with mixup augmentation, achieving substantial robustness improvements on CIFAR-10/100 and TinyImageNet datasets.
Despite being robust to small amounts of label noise, convolutional neural networks trained with stochastic gradient methods have been shown to easily fit random labels. When there are a mixture of correct and mislabelled targets, networks tend to fit the former before the latter. This suggests using a suitable two-component mixture model as an unsupervised generative model of sample loss values during training to allow online estimation of the probability that a sample is mislabelled. Specifically, we propose a beta mixture to estimate this probability and correct the loss by relying on the network prediction (the so-called bootstrapping loss). We further adapt mixup augmentation to drive our approach a step further. Experiments on CIFAR-10/100 and TinyImageNet demonstrate a robustness to label noise that substantially outperforms recent state-of-the-art. Source code is available at https://git.io/fjsvE