Effects of label noise on the classification of outlier observations
This addresses robustness issues in outlier detection for classification tasks, but it is incremental as it tests an existing method in a new scenario.
The study examined how adding label noise to training data affects the BCOPS algorithm's ability to classify outlier observations, finding that even small amounts of noise significantly degrade model performance.
This study investigates the impact of adding noise to the training set classes in classification tasks using the BCOPS algorithm (Balanced and Conformal Optimized Prediction Sets), proposed by Guan & Tibshirani (2022). The BCOPS algorithm is an application of conformal prediction combined with a machine learning method to construct prediction sets such that the probability of the true class being included in the prediction set for a test observation meets a specified coverage guarantee. An observation is considered an outlier if its true class is not present in the training set. The study employs both synthetic and real datasets and conducts experiments to evaluate the prediction abstention rate for outlier observations and the model's robustness in this previously untested scenario. The results indicate that the addition of noise, even in small amounts, can have a significant effect on model performance.