V-CECE: Visual Counterfactual Explanations via Conceptual Edits
This addresses the need for interpretable AI in computer vision by providing a method to explain classifier decisions, though it is incremental as it builds on existing counterfactual and diffusion model techniques.
The paper tackles the problem of generating counterfactual explanations for black-box classifiers by proposing a plug-and-play framework that uses a pre-trained diffusion model to produce human-level explanations without training, achieving results validated through human evaluation with CNN, ViT, and LVLM classifiers.
Recent black-box counterfactual generation frameworks fail to take into account the semantic content of the proposed edits, while relying heavily on training to guide the generation process. We propose a novel, plug-and-play black-box counterfactual generation framework, which suggests step-by-step edits based on theoretical guarantees of optimal edits to produce human-level counterfactual explanations with zero training. Our framework utilizes a pre-trained image editing diffusion model, and operates without access to the internals of the classifier, leading to an explainable counterfactual generation process. Throughout our experimentation, we showcase the explanatory gap between human reasoning and neural model behavior by utilizing both Convolutional Neural Network (CNN), Vision Transformer (ViT) and Large Vision Language Model (LVLM) classifiers, substantiated through a comprehensive human evaluation.