Unifews: You Need Fewer Operations for Efficient Graph Neural Networks
This work addresses efficiency issues for researchers and practitioners using GNNs on resource-intensive tasks, representing a novel method rather than an incremental improvement.
The paper tackles the problem of high computational overhead in Graph Neural Networks (GNNs) by proposing Unifews, a joint sparsification technique that unifies graph and weight matrix operations, resulting in 10-20x matrix operation reduction and up to 100x acceleration on large-scale graphs while maintaining comparable or better accuracy.
Graph Neural Networks (GNNs) have shown promising performance, but at the cost of resource-intensive operations on graph-scale matrices. To reduce computational overhead, previous studies attempt to sparsify the graph or network parameters, but with limited flexibility and precision boundaries. In this work, we propose Unifews, a joint sparsification technique to unify graph and weight matrix operations and enhance GNN learning efficiency. The Unifews design enables adaptive compression across GNN layers with progressively increased sparsity, and is applicable to a variety of architectures with on-the-fly simplification. Theoretically, we establish a novel framework to characterize sparsified GNN learning in view of the graph optimization process, showing that Unifews effectively approximates the learning objective with bounded error and reduced computational overhead. Extensive experiments demonstrate that Unifews achieves efficiency improvements with comparable or better accuracy, including 10-20x matrix operation reduction and up to 100x acceleration on graphs up to billion-edge scale.