Robustness and uncertainty: two complementary aspects of the reliability of the predictions of a classifier
This addresses reliability assessment for machine learning practitioners, but it is incremental as it combines existing approaches.
The paper tackled the problem of assessing the reliability of individual classifier predictions by comparing Robustness Quantification (RQ) and Uncertainty Quantification (UQ), finding they are complementary and that a hybrid approach outperforms both.
We consider two conceptually different approaches for assessing the reliability of the individual predictions of a classifier: Robustness Quantification (RQ) and Uncertainty Quantification (UQ). We compare both approaches on a number of benchmark datasets and show that there is no clear winner between the two, but that they are complementary and can be combined to obtain a hybrid approach that outperforms both RQ and UQ. As a byproduct of our approach, for each dataset, we also obtain an assessment of the relative importance of uncertainty and robustness as sources of unreliability.