Sparse Visual Counterfactual Explanations in Image Space
This addresses the need for more interpretable and robust explanations in computer vision, particularly for detecting spurious features in datasets like ImageNet, though it is incremental in improving existing VCE methods.
The paper tackles the problem of generating visual counterfactual explanations (VCEs) in image space for image classifiers, which often suffer from spurious background changes, and results in sparse VCEs that produce subtle, class-specific modifications.
Visual counterfactual explanations (VCEs) in image space are an important tool to understand decisions of image classifiers as they show under which changes of the image the decision of the classifier would change. Their generation in image space is challenging and requires robust models due to the problem of adversarial examples. Existing techniques to generate VCEs in image space suffer from spurious changes in the background. Our novel perturbation model for VCEs together with its efficient optimization via our novel Auto-Frank-Wolfe scheme yields sparse VCEs which lead to subtle changes specific for the target class. Moreover, we show that VCEs can be used to detect undesired behavior of ImageNet classifiers due to spurious features in the ImageNet dataset.