Trustworthy Prediction with Gaussian Process Knowledge Scores
This provides a tool for practitioners using Gaussian processes to gauge prediction trustworthiness, but it is incremental as it builds on existing GPR methods.
The paper tackles the problem of assessing whether Gaussian process regression predictions are well-informed by data, proposing a knowledge score that quantifies uncertainty reduction; experiments show it improves performance in anomaly detection, extrapolation, and imputation tasks.
Probabilistic models are often used to make predictions in regions of the data space where no observations are available, but it is not always clear whether such predictions are well-informed by previously seen data. In this paper, we propose a knowledge score for predictions from Gaussian process regression (GPR) models that quantifies the extent to which observing data have reduced our uncertainty about a prediction. The knowledge score is interpretable and naturally bounded between 0 and 1. We demonstrate in several experiments that the knowledge score can anticipate when predictions from a GPR model are accurate, and that this anticipation improves performance in tasks such as anomaly detection, extrapolation, and missing data imputation. Source code for this project is available online at https://github.com/KurtButler/GP-knowledge.