Not All Neighbors Matter: Understanding the Impact of Graph Sparsification on GNN Pipelines
This work provides a practical solution for researchers and practitioners working with large-scale GNNs by demonstrating that graph sparsification can substantially improve performance without significant accuracy loss, and often with gains.
This paper investigates graph sparsification as a pre-processing step for Graph Neural Network (GNN) pipelines to address data management and movement bottlenecks in large-scale graphs. They found that sparsification often preserves or improves predictive performance, with random sparsification raising GAT model accuracy by 6.8% on PubMed, and significantly accelerates training and inference, with K-Neighbor sparsifier improving model serving performance on Products by 11.7x with a 0.7% accuracy drop.
As graphs scale to billions of nodes and edges, graph Machine Learning workloads are constrained by the cost of multi-hop traversals over exponentially growing neighborhoods. While various system-level and algorithmic optimizations have been proposed to accelerate Graph Neural Network (GNN) pipelines, data management and movement remain the primary bottlenecks at scale. In this paper, we explore whether graph sparsification, a well-established technique that reduces edges to create sparser neighborhoods, can serve as a lightweight pre-processing step to address these bottlenecks while preserving accuracy on node classification tasks. We develop an extensible experimental framework that enables systematic evaluation of how different sparsification methods affect the performance and accuracy of GNN models. We conduct the first comprehensive study of GNN training and inference on sparsified graphs, revealing several key findings. First, sparsification often preserves or even improves predictive performance. As an example, random sparsification raises the accuracy of the GAT model by 6.8% on the PubMed graph. Second, benefits increase with scale, substantially accelerating both training and inference. Our results show that the K-Neighbor sparsifier improves model serving performance on the Products graph by 11.7x with only a 0.7% accuracy drop. Importantly, we find that the computational overhead of sparsification is quickly amortized, making it practical for very large graphs.