Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
This addresses the challenge of deploying low-latency MLPs in scenarios where graph structures are unavailable, offering an incremental improvement over previous edge-based methods.
The paper tackles the problem of distilling Graph Neural Networks (GNNs) to Multilayer Perceptrons (MLPs) for graph tasks without requiring graph edges, proposing Prototype-Guided Knowledge Distillation (PGKD) to capture structural information, with experimental results showing effectiveness and robustness on benchmarks.
Distilling high-accuracy Graph Neural Networks (GNNs) to low-latency multilayer perceptions (MLPs) on graph tasks has become a hot research topic. However, conventional MLP learning relies almost exclusively on graph nodes and fails to effectively capture the graph structural information. Previous methods address this issue by processing graph edges into extra inputs for MLPs, but such graph structures may be unavailable for various scenarios. To this end, we propose Prototype-Guided Knowledge Distillation (PGKD), which does not require graph edges (edge-free setting) yet learns structure-aware MLPs. Our insight is to distill graph structural information from GNNs. Specifically, we first employ the class prototypes to analyze the impact of graph structures on GNN teachers, and then design two losses to distill such information from GNNs to MLPs. Experimental results on popular graph benchmarks demonstrate the effectiveness and robustness of the proposed PGKD.