Constrained Instance and Class Reweighting for Robust Learning under Label Noise
This addresses the issue of robust learning under label noise for machine learning practitioners, though it appears incremental as it builds on existing label smoothing heuristics.
The paper tackles the problem of label noise degrading deep neural network performance by proposing a constrained optimization approach to assign importance weights to instances and class labels, achieving considerable performance gains on benchmark datasets.
Deep neural networks have shown impressive performance in supervised learning, enabled by their ability to fit well to the provided training data. However, their performance is largely dependent on the quality of the training data and often degrades in the presence of noise. We propose a principled approach for tackling label noise with the aim of assigning importance weights to individual instances and class labels. Our method works by formulating a class of constrained optimization problems that yield simple closed form updates for these importance weights. The proposed optimization problems are solved per mini-batch which obviates the need of storing and updating the weights over the full dataset. Our optimization framework also provides a theoretical perspective on existing label smoothing heuristics for addressing label noise (such as label bootstrapping). We evaluate our method on several benchmark datasets and observe considerable performance gains in the presence of label noise.