MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
This work is significant for researchers and practitioners working with deep learning models on noisy or imbalanced datasets, offering an incremental improvement in sample reweighting techniques.
This paper addresses the problem of deep neural networks overfitting to biased training data (label noise, class imbalance) by proposing MetaInfoNet. This method automatically learns effective representations as inputs for a meta weighting network using an information bottleneck strategy, outperforming state-of-the-art methods on benchmark datasets.
Deep neural networks have been shown to easily overfit to biased training data with label noise or class imbalance. Meta-learning algorithms are commonly designed to alleviate this issue in the form of sample reweighting, by learning a meta weighting network that takes training losses as inputs to generate sample weights. In this paper, we advocate that choosing proper inputs for the meta weighting network is crucial for desired sample weights in a specific task, while training loss is not always the correct answer. In view of this, we propose a novel meta-learning algorithm, MetaInfoNet, which automatically learns effective representations as inputs for the meta weighting network by emphasizing task-related information with an information bottleneck strategy. Extensive experimental results on benchmark datasets with label noise or class imbalance validate that MetaInfoNet is superior to many state-of-the-art methods.