Human uncertainty makes classification more robust
This addresses the issue of limited robustness in AI classifiers for real-world applications, though it is incremental as it builds on existing methods with new data.
The paper tackles the problem of deep neural networks' poor generalization and adversarial robustness by training them with full label distributions that capture human perceptual uncertainty, resulting in improved generalization to out-of-distribution datasets and enhanced robustness to adversarial attacks.
The classification performance of deep neural networks has begun to asymptote at near-perfect levels. However, their ability to generalize outside the training set and their robustness to adversarial attacks have not. In this paper, we make progress on this problem by training with full label distributions that reflect human perceptual uncertainty. We first present a new benchmark dataset which we call CIFAR10H, containing a full distribution of human labels for each image of the CIFAR10 test set. We then show that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on our dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.