Evaluating Explainability for Graph Neural Networks
This work addresses the problem of unreliable evaluation for GNN explanations, which is crucial for researchers and practitioners in machine learning, but it is incremental as it builds on existing explainability methods by providing better evaluation tools.
The paper tackles the challenge of evaluating explainability for graph neural networks (GNNs) by introducing ShapeGGen, a synthetic graph data generator that creates benchmark datasets with ground-truth explanations, and integrates it into an open-source library called GraphXAI, which includes datasets, tools, and metrics for benchmarking GNN explainability methods.
As post hoc explanations are increasingly used to understand the behavior of graph neural networks (GNNs), it becomes crucial to evaluate the quality and reliability of GNN explanations. However, assessing the quality of GNN explanations is challenging as existing graph datasets have no or unreliable ground-truth explanations for a given task. Here, we introduce a synthetic graph data generator, ShapeGGen, which can generate a variety of benchmark datasets (e.g., varying graph sizes, degree distributions, homophilic vs. heterophilic graphs) accompanied by ground-truth explanations. Further, the flexibility to generate diverse synthetic datasets and corresponding ground-truth explanations allows us to mimic the data generated by various real-world applications. We include ShapeGGen and several real-world graph datasets into an open-source graph explainability library, GraphXAI. In addition to synthetic and real-world graph datasets with ground-truth explanations, GraphXAI provides data loaders, data processing functions, visualizers, GNN model implementations, and evaluation metrics to benchmark the performance of GNN explainability methods.