Adaptive Sample Selection for Robust Learning under Label Noise
This work addresses the challenge of robust learning under label noise for machine learning practitioners, offering a more practical solution by removing dependencies on noise rates and thresholds, though it is incremental in improving existing sample selection methods.
The paper tackles the problem of deep neural networks overfitting to noisy labels by proposing an adaptive sample selection strategy that uses only batch statistics, eliminating the need for noise rate information or clean data. The algorithm demonstrates effectiveness on benchmark datasets without additional hyperparameters.
Deep Neural Networks (DNNs) have been shown to be susceptible to memorization or overfitting in the presence of noisily-labelled data. For the problem of robust learning under such noisy data, several algorithms have been proposed. A prominent class of algorithms rely on sample selection strategies wherein, essentially, a fraction of samples with loss values below a certain threshold are selected for training. These algorithms are sensitive to such thresholds, and it is difficult to fix or learn these thresholds. Often, these algorithms also require information such as label noise rates which are typically unavailable in practice. In this paper, we propose an adaptive sample selection strategy that relies only on batch statistics of a given mini-batch to provide robustness against label noise. The algorithm does not have any additional hyperparameters for sample selection, does not need any information on noise rates and does not need access to separate data with clean labels. We empirically demonstrate the effectiveness of our algorithm on benchmark datasets.