Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI
This toolkit addresses the need for standardized and accessible uncertainty quantification methods in AI development, though it is incremental as it builds on existing practices without introducing new paradigms.
The paper introduces Uncertainty Quantification 360 (UQ360), an open-source Python toolkit designed to streamline and enhance the quantification, evaluation, improvement, and communication of uncertainty in AI models, with additional educational tools to support researchers and developers.
In this paper, we describe an open source Python toolkit named Uncertainty Quantification 360 (UQ360) for the uncertainty quantification of AI models. The goal of this toolkit is twofold: first, to provide a broad range of capabilities to streamline as well as foster the common practices of quantifying, evaluating, improving, and communicating uncertainty in the AI application development lifecycle; second, to encourage further exploration of UQ's connections to other pillars of trustworthy AI such as fairness and transparency through the dissemination of latest research and education materials. Beyond the Python package (\url{https://github.com/IBM/UQ360}), we have developed an interactive experience (\url{http://uq360.mybluemix.net}) and guidance materials as educational tools to aid researchers and developers in producing and communicating high-quality uncertainties in an effective manner.