Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification
This provides a practical solution for researchers and practitioners in AI and ML to improve reliability in critical applications, though it is incremental as it builds on existing UQ methods.
The authors tackled the lack of a unified tool for evaluating and integrating uncertainty quantification methods in deep learning by introducing Torch-Uncertainty, a PyTorch-based framework that benchmarks diverse UQ techniques across classification, segmentation, and regression tasks.
Deep Neural Networks (DNNs) have demonstrated remarkable performance across various domains, including computer vision and natural language processing. However, they often struggle to accurately quantify the uncertainty of their predictions, limiting their broader adoption in critical real-world applications. Uncertainty Quantification (UQ) for Deep Learning seeks to address this challenge by providing methods to improve the reliability of uncertainty estimates. Although numerous techniques have been proposed, a unified tool offering a seamless workflow to evaluate and integrate these methods remains lacking. To bridge this gap, we introduce Torch-Uncertainty, a PyTorch and Lightning-based framework designed to streamline DNN training and evaluation with UQ techniques and metrics. In this paper, we outline the foundational principles of our library and present comprehensive experimental results that benchmark a diverse set of UQ methods across classification, segmentation, and regression tasks. Our library is available at https://github.com/ENSTA-U2IS-AI/Torch-Uncertainty