CATGNN: Cost-Efficient and Scalable Distributed Training for Graph Neural Networks
This addresses the problem of scaling GNN training to large graphs for researchers and practitioners using commodity hardware, representing a strong incremental improvement in distributed systems.
The paper tackles the challenge of training graph neural networks on large-scale graphs with limited memory by proposing CATGNN, a distributed training system that uses streaming input and a novel partitioning algorithm, enabling handling of billion-scale graphs and reducing replication factor by 50% on average.
Graph neural networks have been shown successful in recent years. While different GNN architectures and training systems have been developed, GNN training on large-scale real-world graphs still remains challenging. Existing distributed systems load the entire graph in memory for graph partitioning, requiring a huge memory space to process large graphs and thus hindering GNN training on such large graphs using commodity workstations. In this paper, we propose CATGNN, a cost-efficient and scalable distributed GNN training system which focuses on scaling GNN training to billion-scale or larger graphs under limited computational resources. Among other features, it takes a stream of edges as input, instead of loading the entire graph in memory, for partitioning. We also propose a novel streaming partitioning algorithm named SPRING for distributed GNN training. We verify the correctness and effectiveness of CATGNN with SPRING on 16 open datasets. In particular, we demonstrate that CATGNN can handle the largest publicly available dataset with limited memory, which would have been infeasible without increasing the memory space. SPRING also outperforms state-of-the-art partitioning algorithms significantly, with a 50% reduction in replication factor on average.