GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs
This work addresses the need for efficient continuous learning on dynamic graphs, which is crucial for applications like social networks or recommendation systems, though it is incremental as it builds upon existing frameworks like DGL and PyTorch.
The paper tackles the problem of training Graph Neural Networks (GNNs) on dynamic graphs, which are common in real-world applications but not well-supported by existing frameworks, by introducing GNNFlow, a distributed framework that achieves up to 21.1x faster continuous learning compared to current systems.
Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are dynamic and contain time domain information. We introduce GNNFlow, a distributed framework that enables efficient continuous temporal graph representation learning on dynamic graphs on multi-GPU machines. GNNFlow introduces an adaptive time-indexed block-based data structure that effectively balances memory usage with graph update and sampling operation efficiency. It features a hybrid GPU-CPU graph data placement for rapid GPU-based temporal neighborhood sampling and kernel optimizations for enhanced sampling processes. A dynamic GPU cache for node and edge features is developed to maximize cache hit rates through reuse and restoration strategies. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. We implement GNNFlow based on DGL and PyTorch. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.