LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
It addresses the need for interpretable models in domains like healthcare and scientific research, offering a novel method for counterfactual explanation that is model-agnostic and handles human-in-the-loop systems, though it is incremental in improving existing explanation techniques.
The paper tackles the problem of generating reliable counterfactual explanations for ML/AI models in high-stakes domains by proposing LeapFactual, a method based on conditional flow matching that overcomes limitations like gradient vanishing and discontinuous latent spaces, resulting in accurate and in-distribution counterfactuals that can be used as new training data to enhance models.
The growing integration of machine learning (ML) and artificial intelligence (AI) models into high-stakes domains such as healthcare and scientific research calls for models that are not only accurate but also interpretable. Among the existing explainable methods, counterfactual explanations offer interpretability by identifying minimal changes to inputs that would alter a model's prediction, thus providing deeper insights. However, current counterfactual generation methods suffer from critical limitations, including gradient vanishing, discontinuous latent spaces, and an overreliance on the alignment between learned and true decision boundaries. To overcome these limitations, we propose LeapFactual, a novel counterfactual explanation algorithm based on conditional flow matching. LeapFactual generates reliable and informative counterfactuals, even when true and learned decision boundaries diverge. Following a model-agnostic approach, LeapFactual is not limited to models with differentiable loss functions. It can even handle human-in-the-loop systems, expanding the scope of counterfactual explanations to domains that require the participation of human annotators, such as citizen science. We provide extensive experiments on benchmark and real-world datasets showing that LeapFactual generates accurate and in-distribution counterfactual explanations that offer actionable insights. We observe, for instance, that our reliable counterfactual samples with labels aligning to ground truth can be beneficially used as new training data to enhance the model. The proposed method is broadly applicable and enhances both scientific knowledge discovery and non-expert interpretability.