Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization
This work addresses the robustness problem for GNNs in inductive learning settings, offering a novel approach to adversarial training that improves both adversarial and clean performance, though it is incremental in advancing existing adversarial training methods for graphs.
The paper tackles the vulnerability of graph neural networks (GNNs) to adversarial topology perturbations by proposing a new concept called graph subspace energy (GSE) as an indicator of robustness and an adversarial training method, AT-GSE, which consistently outperforms state-of-the-art methods in adversarial accuracy across various datasets and even achieves superior clean accuracy on non-perturbed graphs.
Despite impressive capability in learning over graph-structured data, graph neural networks (GNN) suffer from adversarial topology perturbation in both training and inference phases. While adversarial training has demonstrated remarkable effectiveness in image classification tasks, its suitability for GNN models has been doubted until a recent advance that shifts the focus from transductive to inductive learning. Still, GNN robustness in the inductive setting is under-explored, and it calls for deeper understanding of GNN adversarial training. To this end, we propose a new concept of graph subspace energy (GSE) -- a generalization of graph energy that measures graph stability -- of the adjacency matrix, as an indicator of GNN robustness against topology perturbations. To further demonstrate the effectiveness of such concept, we propose an adversarial training method with the perturbed graphs generated by maximizing the GSE regularization term, referred to as AT-GSE. To deal with the local and global topology perturbations raised respectively by LRBCD and PRBCD, we employ randomized SVD (RndSVD) and Nystrom low-rank approximation to favor the different aspects of the GSE terms. An extensive set of experiments shows that AT-GSE outperforms consistently the state-of-the-art GNN adversarial training methods over different homophily and heterophily datasets in terms of adversarial accuracy, whilst more surprisingly achieving a superior clean accuracy on non-perturbed graphs.