Explaining and visualizing black-box models through counterfactual paths
This work addresses the need for transparency in machine learning models, particularly for applications like medical data, by providing a new method to generate explanations, though it appears incremental by building on existing XAI techniques with a graph-based extension.
The paper tackled the problem of explaining black-box models by proposing a novel approach using counterfactual paths generated through conditional permutations of features, which introduces an additional graph dimension to current XAI methods for improved explanation and visualization.
Explainable AI (XAI) is an increasingly important area of machine learning research, which aims to make black-box models transparent and interpretable. In this paper, we propose a novel approach to XAI that uses the so-called counterfactual paths generated by conditional permutations of features. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge. Counterfactual paths introduce an additional graph dimension to current XAI methods in both explaining and visualizing black-box models. Experiments with synthetic and medical data demonstrate the practical applicability of our approach.