Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
This addresses the need for trustworthy predictions in critical domains like medicine, offering a post-training audit tool that is incremental by building on existing uncertainty estimation methods.
The paper tackles the problem of assessing the reliability of machine learning predictions in high-stakes applications by introducing resampling uncertainty estimation (RUE), an algorithm that audits pointwise reliability after training, showing it effectively detects inaccurate predictions and creates competitive predictive distributions compared to state-of-the-art methods.
To use machine learning in high stakes applications (e.g. medicine), we need tools for building confidence in the system and evaluating whether it is reliable. Methods to improve model reliability often require new learning algorithms (e.g. using Bayesian inference to obtain uncertainty estimates). An alternative is to audit a model after it is trained. In this paper, we describe resampling uncertainty estimation (RUE), an algorithm to audit the pointwise reliability of predictions. Intuitively, RUE estimates the amount that a prediction would change if the model had been fit on different training data. The algorithm uses the gradient and Hessian of the model's loss function to create an ensemble of predictions. Experimentally, we show that RUE more effectively detects inaccurate predictions than existing tools for auditing reliability subsequent to training. We also show that RUE can create predictive distributions that are competitive with state-of-the-art methods like Monte Carlo dropout, probabilistic backpropagation, and deep ensembles, but does not depend on specific algorithms at train-time like these methods do.