Prediction Confidence from Neighbors
This addresses the need for human supervision in critical ML applications by providing a confidence metric, though it is incremental as it builds on existing distance-based methods.
The paper tackles the problem of ML models failing on out-of-distribution samples by proposing feature space distance as a confidence measure, showing it correlates with prediction error to enable trust or rejection of predictions based on distance to training samples.
The inability of Machine Learning (ML) models to successfully extrapolate correct predictions from out-of-distribution (OoD) samples is a major hindrance to the application of ML in critical applications. Until the generalization ability of ML methods is improved it is necessary to keep humans in the loop. The need for human supervision can only be reduced if it is possible to determining a level of confidence in predictions, which can be used to either ask for human assistance or to abstain from making predictions. We show that feature space distance is a meaningful measure that can provide confidence in predictions. The distance between unseen samples and nearby training samples proves to be correlated to the prediction error of unseen samples. Depending on the acceptable degree of error, predictions can either be trusted or rejected based on the distance to training samples. %Additionally, a novelty threshold can be used to decide whether a sample is worth adding to the training set. This enables earlier and safer deployment of models in critical applications and is vital for deploying models under ever-changing conditions.