Early Stopping Against Label Noise Without Validation Data
This addresses the challenge of model selection without validation data in noisy-label scenarios, offering a practical solution for applications where data is scarce or noisy, though it is incremental as it builds on early stopping and noisy label learning methods.
The paper tackles the problem of early stopping in deep learning when label noise is present and validation data is unavailable, proposing the Label Wave method that tracks prediction fluctuations on the training set to halt training before overfitting to mislabeled data, with experiments showing effectiveness across various settings and improved performance for existing noisy label methods.
Early stopping methods in deep learning face the challenge of balancing the volume of training and validation data, especially in the presence of label noise. Concretely, sparing more data for validation from training data would limit the performance of the learned model, yet insufficient validation data could result in a sub-optimal selection of the desired model. In this paper, we propose a novel early stopping method called Label Wave, which does not require validation data for selecting the desired model in the presence of label noise. It works by tracking the changes in the model's predictions on the training set during the training process, aiming to halt training before the model unduly fits mislabeled data. This method is empirically supported by our observation that minimum fluctuations in predictions typically occur at the training epoch before the model excessively fits mislabeled data. Through extensive experiments, we show both the effectiveness of the Label Wave method across various settings and its capability to enhance the performance of existing methods for learning with noisy labels.