Algorithmic Concept-based Explainable Reasoning
This addresses the problem of interpretability in GNNs for researchers and practitioners in machine learning and graph-based applications, though it is incremental as it applies existing concept-based explanation methods to GNNs.
The paper tackles the lack of explainability in graph neural networks (GNNs) applied to graph algorithms and combinatorial optimization by introducing concept-bottleneck GNNs, which learn concepts and extract propositional formulas, achieving comparative performance with state-of-the-art models in three case studies.
Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of algorithms when preconditions are not satisfied, or reusing learned models when sufficient training data is not available or can't be generated. Unfortunately, a key hindrance of these approaches is their lack of explainability, since GNNs are black-box models that cannot be interpreted directly. In this work, we address this limitation by applying existing work on concept-based explanations to GNN models. We introduce concept-bottleneck GNNs, which rely on a modification to the GNN readout mechanism. Using three case studies we demonstrate that: (i) our proposed model is capable of accurately learning concepts and extracting propositional formulas based on the learned concepts for each target class; (ii) our concept-based GNN models achieve comparative performance with state-of-the-art models; (iii) we can derive global graph concepts, without explicitly providing any supervision on graph-level concepts.