Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNs
This addresses the reliability of explanation tools for GNNs, which is crucial for their adoption in high-stakes domains, though it is incremental in refining existing concepts rather than introducing a new paradigm.
The paper tackles the problem of defining and achieving faithfulness in explanations for Graph Neural Networks (GNNs), showing that existing faithfulness metrics are not interchangeable and that optimizing for faithfulness can be uninformative for certain architectures, while linking faithfulness to out-of-distribution generalization.
As Graph Neural Networks (GNNs) become more pervasive, it becomes paramount to build reliable tools for explaining their predictions. A core desideratum is that explanations are \textit{faithful}, \ie that they portray an accurate picture of the GNN's reasoning process. However, a number of different faithfulness metrics exist, begging the question of what is faithfulness exactly and how to achieve it. We make three key contributions. We begin by showing that \textit{existing metrics are not interchangeable} -- \ie explanations attaining high faithfulness according to one metric may be unfaithful according to others -- and can systematically ignore important properties of explanations. We proceed to show that, surprisingly, \textit{optimizing for faithfulness is not always a sensible design goal}. Specifically, we prove that for injective regular GNN architectures, perfectly faithful explanations are completely uninformative. This does not apply to modular GNNs, such as self-explainable and domain-invariant architectures, prompting us to study the relationship between architectural choices and faithfulness. Finally, we show that \textit{faithfulness is tightly linked to out-of-distribution generalization}, in that simply ensuring that a GNN can correctly recognize the domain-invariant subgraph, as prescribed by the literature, does not guarantee that it is invariant unless this subgraph is also faithful.The code is publicly available on GitHub