Searching for Robustness: Loss Learning for Noisy Classification Tasks
This addresses the challenge of noisy labels in machine learning classification, offering a reusable solution without requiring special training procedures, though it is incremental as it builds on existing loss function optimization approaches.
The authors tackled the problem of label noise in classification tasks by automatically constructing white-box loss functions that are robust to such noise, resulting in a plug-and-play module that performs favorably compared to previous methods on various datasets with synthetic and real noise.
We present a "learning to learn" approach for automatically constructing white-box classification loss functions that are robust to label noise in the training data. We parameterize a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training dataset and architecture combinations. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective noise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture. The efficacy of our method is demonstrated on a variety of datasets with both synthetic and real label noise, where we compare favourably to previous work.