Robustness quantification: a new method for assessing the reliability of the predictions of a classifier
This work addresses the challenge of reliable prediction assessment in machine learning, particularly for classifiers with limited or non-representative data, representing an incremental advancement in uncertainty and robustness methods.
The paper tackles the problem of assessing the reliability of individual predictions from generative probabilistic classifiers, introducing a new method called robustness quantification that performs well even with small training sets from shifted distributions.
Based on existing ideas in the field of imprecise probabilities, we present a new approach for assessing the reliability of the individual predictions of a generative probabilistic classifier. We call this approach robustness quantification, compare it to uncertainty quantification, and demonstrate that it continues to work well even for classifiers that are learned from small training sets that are sampled from a shifted distribution.