An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
This work addresses robustness and uncertainty problems in deep learning for image classification, but it is incremental as it focuses on evaluating existing regularization methods rather than proposing new solutions.
The paper tackles the lack of robustness and improper uncertainty estimation in deep neural networks by empirically evaluating state-of-the-art regularization methods on image classifiers, finding that certain methods can serve as strong baselines for these issues.
Despite apparent human-level performances of deep neural networks (DNN), they behave fundamentally differently from humans. They easily change predictions when small corruptions such as blur and noise are applied on the input (lack of robustness), and they often produce confident predictions on out-of-distribution samples (improper uncertainty measure). While a number of researches have aimed to address those issues, proposed solutions are typically expensive and complicated (e.g. Bayesian inference and adversarial training). Meanwhile, many simple and cheap regularization methods have been developed to enhance the generalization of classifiers. Such regularization methods have largely been overlooked as baselines for addressing the robustness and uncertainty issues, as they are not specifically designed for that. In this paper, we provide extensive empirical evaluations on the robustness and uncertainty estimates of image classifiers (CIFAR-100 and ImageNet) trained with state-of-the-art regularization methods. Furthermore, experimental results show that certain regularization methods can serve as strong baseline methods for robustness and uncertainty estimation of DNNs.