A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
This work addresses the challenge of noisy labels in datasets for practical classification applications, offering an incremental improvement over existing methods.
The paper tackles the problem of training deep classifiers under input-dependent label noise by proposing a simple probabilistic method that uses a temperature-parameterized softmax approximation, resulting in improved accuracy, log-likelihood, and calibration on image classification benchmarks and a more than 1% increase in mean IoU for image segmentation tasks.
Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. We assume an underlying heteroscedastic generative process for noisy labels. To make gradient based training feasible we use a temperature parameterized softmax as a smooth approximation to the assumed generative process. We illustrate that the softmax temperature controls a bias-variance trade-off for the approximation. By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.