Redundancy-Free Computation Graphs for Graph Neural Networks
This addresses inefficiency in GNN training and inference for machine learning practitioners, offering a novel optimization method rather than an incremental improvement.
The paper tackles the problem of repeated and inefficient computations in Graph Neural Networks (GNNs) due to shared neighbors, proposing Hierarchically Aggregated computation Graphs (HAGs) to eliminate redundancy, resulting in up to 2.8x increased training throughput and up to 6.3x reduction in aggregations while maintaining accuracy.
Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different nodes, this leads to repeated and inefficient computations. We propose Hierarchically Aggregated computation Graphs (HAGs), a new GNN graph representation that explicitly avoids redundancy by managing intermediate aggregation results hierarchically, eliminating repeated computations and unnecessary data transfers in GNN training and inference. We introduce an accurate cost function to quantitatively evaluate the runtime performance of different HAGs and use a novel HAG search algorithm to find optimized HAGs. Experiments show that the HAG representation significantly outperforms the standard GNN graph representation by increasing the end-to-end training throughput by up to 2.8x and reducing the aggregations and data transfers in GNN training by up to 6.3x and 5.6x, while maintaining the original model accuracy.