Detecting Label Noise via Leave-One-Out Cross-Validation
This addresses label noise issues in scientific data analysis, but it is incremental as it builds on existing Gaussian process methods.
The paper tackles the problem of detecting and correcting noisy labels in regression tasks by using Gaussian process regression with a heteroscedastic noise model and leave-one-out cross-validation, showing improved model performance on synthetic and real-world datasets.
We present a simple algorithm for identifying and correcting real-valued noisy labels from a mixture of clean and corrupted sample points using Gaussian process regression. A heteroscedastic noise model is employed, in which additive Gaussian noise terms with independent variances are associated with each and all of the observed labels. Optimizing the noise model using maximum likelihood estimation leads to the containment of the GPR model's predictive error by the posterior standard deviation in leave-one-out cross-validation. A multiplicative update scheme is proposed for solving the maximum likelihood estimation problem under non-negative constraints. While we provide proof of convergence for certain special cases, the multiplicative scheme has empirically demonstrated monotonic convergence behavior in virtually all our numerical experiments. We show that the presented method can pinpoint corrupted sample points and lead to better regression models when trained on synthetic and real-world scientific data sets.