Verifying Relational Explanations: A Probabilistic Approach
This addresses the challenge of verifying explanations for Graph Neural Networks, which is crucial for trust and debugging in domains like social networks or bioinformatics, but the approach is incremental as it builds on GNNExplainer.
The paper tackles the problem of verifying relational explanations from GNNExplainer, which are hard to assess due to complexity and scalability issues, by developing a probabilistic approach that estimates explanation uncertainty through counterfactual examples and factor graph models, showing reliable uncertainty estimation on several datasets.
Explanations on relational data are hard to verify since the explanation structures are more complex (e.g. graphs). To verify interpretable explanations (e.g. explanations of predictions made in images, text, etc.), typically human subjects are used since it does not necessarily require a lot of expertise. However, to verify the quality of a relational explanation requires expertise and is hard to scale-up. GNNExplainer is arguably one of the most popular explanation methods for Graph Neural Networks. In this paper, we develop an approach where we assess the uncertainty in explanations generated by GNNExplainer. Specifically, we ask the explainer to generate explanations for several counterfactual examples. We generate these examples as symmetric approximations of the relational structure in the original data. From these explanations, we learn a factor graph model to quantify uncertainty in an explanation. Our results on several datasets show that our approach can help verify explanations from GNNExplainer by reliably estimating the uncertainty of a relation specified in the explanation.