Deep Learning with Label Noise: A Hierarchical Approach
This addresses the issue of label noise in deep learning for practitioners, offering a simple, incremental improvement over existing methods.
The paper tackles the problem of deep neural networks being susceptible to label noise by proposing a hierarchical approach that incorporates a label hierarchy during training without altering network architecture or optimization. The result is improved performance over regular networks in noisy settings, achieving state-of-the-art results when combined with pre-trained models on real-world datasets.
Deep neural networks are susceptible to label noise. Existing methods to improve robustness, such as meta-learning and regularization, usually require significant change to the network architecture or careful tuning of the optimization procedure. In this work, we propose a simple hierarchical approach that incorporates a label hierarchy when training the deep learning models. Our approach requires no change of the network architecture or the optimization procedure. We investigate our hierarchical network through a wide range of simulated and real datasets and various label noise types. Our hierarchical approach improves upon regular deep neural networks in learning with label noise. Combining our hierarchical approach with pre-trained models achieves state-of-the-art performance in real-world noisy datasets.