Learning to Purify Noisy Labels via Meta Soft Label Corrector
This work addresses the challenge of noisy labels in deep learning, which is a critical issue for practitioners in fields like computer vision and natural language processing, by providing an adaptive, model-agnostic solution that improves over existing incremental label correction strategies.
The paper tackles the problem of deep neural networks overfitting to training data with noisy labels by proposing a meta-learning model that automatically corrects labels using meta-gradient descent, eliminating the need for pre-defined rules or manual hyper-parameters. It demonstrates superior performance in synthetic and real-world noisy label scenarios compared to state-of-the-art methods.
Recent deep neural networks (DNNs) can easily overfit to biased training data with noisy labels. Label correction strategy is commonly used to alleviate this issue by designing a method to identity suspected noisy labels and then correct them. Current approaches to correcting corrupted labels usually need certain pre-defined label correction rules or manually preset hyper-parameters. These fixed settings make it hard to apply in practice since the accurate label correction usually related with the concrete problem, training data and the temporal information hidden in dynamic iterations of training process. To address this issue, we propose a meta-learning model which could estimate soft labels through meta-gradient descent step under the guidance of noise-free meta data. By viewing the label correction procedure as a meta-process and using a meta-learner to automatically correct labels, we could adaptively obtain rectified soft labels iteratively according to current training problems without manually preset hyper-parameters. Besides, our method is model-agnostic and we can combine it with any other existing model with ease. Comprehensive experiments substantiate the superiority of our method in both synthetic and real-world problems with noisy labels compared with current SOTA label correction strategies.