Explainable AI for Natural Adversarial Images
This work addresses the vulnerability of image classifiers to adversarial attacks by improving human oversight through explainable AI, though it is incremental as it builds on prior findings about human assumptions.
The study tackled the problem of humans predicting AI errors on adversarial images by evaluating explainable AI methods, finding that saliency maps and examples help catch errors but are not additive, with saliency maps being more effective.
Adversarial images highlight how vulnerable modern image classifiers are to perturbations outside of their training set. Human oversight might mitigate this weakness, but depends on humans understanding the AI well enough to predict when it is likely to make a mistake. In previous work we have found that humans tend to assume that the AI's decision process mirrors their own. Here we evaluate if methods from explainable AI can disrupt this assumption to help participants predict AI classifications for adversarial and standard images. We find that both saliency maps and examples facilitate catching AI errors, but their effects are not additive, and saliency maps are more effective than examples.