How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
This work addresses noisy graph learning for graph neural network users, offering a practical recipe for attention design based on graph characteristics, though it is incremental as it builds on existing graph attention models.
The paper tackles the problem of graph attention mechanisms in noisy graphs by proposing SuperGAT, a self-supervised graph attention network that uses edge prediction to learn more expressive attention, resulting in improved performance over baselines on 15 out of 17 real-world datasets.
Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.