SURE: SUrvey REcipes for building reliable and robust deep networks
This work addresses the need for reliable and robust uncertainty estimation in deep learning, particularly for real-world scenarios involving data corruption and label noise, though it is incremental as it consolidates existing techniques.
The paper tackles the problem of improving uncertainty estimation in deep neural networks for image classification, resulting in the SURE approach that consistently outperforms individual techniques and achieves state-of-the-art performance on datasets like Animal-10N and Food-101N with noisy labels.
In this paper, we revisit techniques for uncertainty estimation within deep neural networks and consolidate a suite of techniques to enhance their reliability. Our investigation reveals that an integrated application of diverse techniques--spanning model regularization, classifier and optimization--substantially improves the accuracy of uncertainty predictions in image classification tasks. The synergistic effect of these techniques culminates in our novel SURE approach. We rigorously evaluate SURE against the benchmark of failure prediction, a critical testbed for uncertainty estimation efficacy. Our results showcase a consistently better performance than models that individually deploy each technique, across various datasets and model architectures. When applied to real-world challenges, such as data corruption, label noise, and long-tailed class distribution, SURE exhibits remarkable robustness, delivering results that are superior or on par with current state-of-the-art specialized methods. Particularly on Animal-10N and Food-101N for learning with noisy labels, SURE achieves state-of-the-art performance without any task-specific adjustments. This work not only sets a new benchmark for robust uncertainty estimation but also paves the way for its application in diverse, real-world scenarios where reliability is paramount. Our code is available at \url{https://yutingli0606.github.io/SURE/}.