Bi-GCN: Binary Graph Convolutional Network
This addresses memory constraints for deploying GNNs in resource-limited settings, though it is an incremental improvement by applying binarization to GNNs.
The paper tackles the memory and computational inefficiency of Graph Neural Networks (GNNs) on large attributed graphs by proposing Bi-GCN, which binarizes parameters and features, achieving ~30x memory reduction and ~47x inference speedup on citation networks while maintaining comparable performance to full-precision baselines.
Graph Neural Networks (GNNs) have achieved tremendous success in graph representation learning. Unfortunately, current GNNs usually rely on loading the entire attributed graph into network for processing. This implicit assumption may not be satisfied with limited memory resources, especially when the attributed graph is large. In this paper, we pioneer to propose a Binary Graph Convolutional Network (Bi-GCN), which binarizes both the network parameters and input node features. Besides, the original matrix multiplications are revised to binary operations for accelerations. According to the theoretical analysis, our Bi-GCN can reduce the memory consumption by an average of ~30x for both the network parameters and input data, and accelerate the inference speed by an average of ~47x, on the citation networks. Meanwhile, we also design a new gradient approximation based back-propagation method to train our Bi-GCN well. Extensive experiments have demonstrated that our Bi-GCN can give a comparable performance compared to the full-precision baselines. Besides, our binarization approach can be easily applied to other GNNs, which has been verified in the experiments.