AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
This work provides a performance optimization for GNN training, which is incremental as it builds on existing sparsity methods to improve kernel efficiency.
The paper tackled the challenge of optimizing Graph Neural Network (GNN) training performance by addressing inefficiencies in prior sparsity-based methods, resulting in a system that achieves up to 6.49x speedup (1.87x on average) over state-of-the-art approaches on GPUs.
Graph neural networks (GNNs) are powerful tools for exploring and learning from graph structures and features. As such, achieving high-performance execution for GNNs becomes crucially important. Prior works have proposed to explore the sparsity (i.e., low density) in the input graph to accelerate GNNs, which uses the full-graph-level or block-level sparsity format. We show that they fail to balance the sparsity benefit and kernel execution efficiency. In this paper, we propose a novel system, referred to as AdaptGear, that addresses the challenge of optimizing GNNs performance by leveraging kernels tailored to the density characteristics at the subgraph level. Meanwhile, we also propose a method that dynamically chooses the optimal set of kernels for a given input graph. Our evaluation shows that AdaptGear can achieve a significant performance improvement, up to $6.49 \times$ ($1.87 \times$ on average), over the state-of-the-art works on two mainstream NVIDIA GPUs across various datasets.