Spectral Augmentation for Self-Supervised Learning on Graphs
This work addresses a domain-specific problem for graph machine learning researchers by providing a principled approach to graph augmentation, though it is incremental as it builds on existing contrastive learning methods.
The paper tackled the problem of unclear invariance in graph contrastive learning by proposing spectral augmentation to guide topology augmentations based on spectral changes, resulting in effective self-supervised representation learning with demonstrated generalization and robustness in experiments.
Graph contrastive learning (GCL), as an emerging self-supervised learning technique on graphs, aims to learn representations via instance discrimination. Its performance heavily relies on graph augmentation to reflect invariant patterns that are robust to small perturbations; yet it still remains unclear about what graph invariance GCL should capture. Recent studies mainly perform topology augmentations in a uniformly random manner in the spatial domain, ignoring its influence on the intrinsic structural properties embedded in the spectral domain. In this work, we aim to find a principled way for topology augmentations by exploring the invariance of graphs from the spectral perspective. We develop spectral augmentation which guides topology augmentations by maximizing the spectral change. Extensive experiments on both graph and node classification tasks demonstrate the effectiveness of our method in self-supervised representation learning. The proposed method also brings promising generalization capability in transfer learning, and is equipped with intriguing robustness property under adversarial attacks. Our study sheds light on a general principle for graph topology augmentation.