Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks
This work addresses safety in model deployment for applications with dire consequences, but it is incremental as it builds on existing uncertainty-aware methods.
The paper tackled the problem of reliably assessing model confidence and predicting errors in deep learning for safety-critical applications by improving the separation between confidence of correct and incorrect predictions using uncertainty-aware Dirichlet networks, and proposed a new criterion for learning true class probability, showing experimental improvements over baselines on image classification tasks.
Reliably assessing model confidence in deep learning and predicting errors likely to be made are key elements in providing safety for model deployment, in particular for applications with dire consequences. In this paper, it is first shown that uncertainty-aware deep Dirichlet neural networks provide an improved separation between the confidence of correct and incorrect predictions in the true class probability (TCP) metric. Second, as the true class is unknown at test time, a new criterion is proposed for learning the true class probability by matching prediction confidence scores while taking imbalance and TCP constraints into account for correct predictions and failures. Experimental results show our method improves upon the maximum class probability (MCP) baseline and predicted TCP for standard networks on several image classification tasks with various network architectures.