Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages
This addresses the computational bottleneck of GNNs for researchers and practitioners working with graph data, though it appears incremental as it builds on existing distillation and contrastive learning techniques.
The paper tackles the computational inefficiency of Graph Neural Networks (GNNs) by proposing LinkDist, a method that distills self-knowledge from connected node pairs into an MLP without message passing, achieving comparable accuracy to GNNs on 8 real-world datasets for node classification.
Nowadays, Graph Neural Networks (GNNs) following the Message Passing paradigm become the dominant way to learn on graphic data. Models in this paradigm have to spend extra space to look up adjacent nodes with adjacency matrices and extra time to aggregate multiple messages from adjacent nodes. To address this issue, we develop a method called LinkDist that distils self-knowledge from connected node pairs into a Multi-Layer Perceptron (MLP) without the need to aggregate messages. Experiment with 8 real-world datasets shows the MLP derived from LinkDist can predict the label of a node without knowing its adjacencies but achieve comparable accuracy against GNNs in the contexts of semi- and full-supervised node classification. Moreover, LinkDist benefits from its Non-Message Passing paradigm that we can also distil self-knowledge from arbitrarily sampled node pairs in a contrastive way to further boost the performance of LinkDist.