Tailored Uncertainty Estimation for Deep Learning Systems
This work addresses the need for trustworthy deep learning systems, particularly in anticipation of regulations like those in the EU, by providing evidence for the technical appropriateness of uncertainty estimation components.
The paper tackles the problem of making deep learning systems more reliable by proposing a framework for tailored uncertainty estimation, which structures requirements, guides method selection, and provides validation strategies to match use case needs.
Uncertainty estimation bears the potential to make deep learning (DL) systems more reliable. Standard techniques for uncertainty estimation, however, come along with specific combinations of strengths and weaknesses, e.g., with respect to estimation quality, generalization abilities and computational complexity. To actually harness the potential of uncertainty quantification, estimators are required whose properties closely match the requirements of a given use case. In this work, we propose a framework that, firstly, structures and shapes these requirements, secondly, guides the selection of a suitable uncertainty estimation method and, thirdly, provides strategies to validate this choice and to uncover structural weaknesses. By contributing tailored uncertainty estimation in this sense, our framework helps to foster trustworthy DL systems. Moreover, it anticipates prospective machine learning regulations that require, e.g., in the EU, evidences for the technical appropriateness of machine learning systems. Our framework provides such evidences for system components modeling uncertainty.