Understanding Classifier Mistakes with Generative Models
This addresses the issue of classifier reliability for users in safety-critical applications, though it is incremental as it builds on existing generative modeling techniques.
The paper tackles the problem of deep neural networks being brittle and prone to errors by using generative models to identify and characterize instances where classifiers fail to generalize, showing that errors occur when features have low probability under the model and developing a detection criteria for likely failures on test data, adversarial samples, and out-of-distribution samples.
Although deep neural networks are effective on supervised learning tasks, they have been shown to be brittle. They are prone to overfitting on their training distribution and are easily fooled by small adversarial perturbations. In this paper, we leverage generative models to identify and characterize instances where classifiers fail to generalize. We propose a generative model of the features extracted by a classifier, and show using rigorous hypothesis testing that errors tend to occur when features are assigned low-probability by our model. From this observation, we develop a detection criteria for samples on which a classifier is likely to fail at test time. In particular, we test against three different sources of classification failures: mistakes made on the test set due to poor model generalization, adversarial samples and out-of-distribution samples. Our approach is agnostic to class labels from the training set which makes it applicable to models trained in a semi-supervised way.