Evaluating Uncertainty Calibration for Open-Set Recognition
This addresses the need for reliable uncertainty estimation in safe robot autonomy, but it is incremental as it evaluates existing methods rather than proposing new ones.
The paper tackles the problem of deep neural networks providing overconfident probabilities on out-of-distribution data in open-set recognition, finding that existing calibration methods are much less effective in this context.
Despite achieving enormous success in predictive accuracy for visual classification problems, deep neural networks (DNNs) suffer from providing overconfident probabilities on out-of-distribution (OOD) data. Yet, accurate uncertainty estimation is crucial for safe and reliable robot autonomy. In this paper, we evaluate popular calibration techniques for open-set conditions in a way that is distinctly different from the conventional evaluation of calibration methods on OOD data. Our results show that closed-set DNN calibration approaches are much less effective for open-set recognition, which highlights the need to develop new DNN calibration methods to address this problem.