Binary classification with corrupted labels
This addresses robustness in classification for scenarios with label noise, but it is incremental as it builds on existing regularization concepts.
The paper tackles the problem of binary classification with corrupted labels, showing that in settings where likelihood maximization is poorly behaved, a small fraction of corrupted labels can act as regularization to improve performance, with error bounds scaling with the square root of the sample size.
In a binary classification problem where the goal is to fit an accurate predictor, the presence of corrupted labels in the training data set may create an additional challenge. However, in settings where likelihood maximization is poorly behaved-for example, if positive and negative labels are perfectly separable-then a small fraction of corrupted labels can improve performance by ensuring robustness. In this work, we establish that in such settings, corruption acts as a form of regularization, and we compute precise upper bounds on estimation error in the presence of corruptions. Our results suggest that the presence of corrupted data points is beneficial only up to a small fraction of the total sample, scaling with the square root of the sample size.