Hierarchical graph sampling based minibatch learning with chain preservation and variance reduction
This work addresses scalability issues in graph neural networks for researchers and practitioners, offering an incremental improvement over existing sampling-based methods.
The paper tackles the problem of scalability in Graph Convolutional Networks (GCNs) during minibatch training by proposing HIS_GCNs, a hierarchical importance sampling method that preserves connectivity and reduces variance, achieving superior accuracy and faster training times in node classification tasks.
Graph sampling-based Graph Convolutional Networks (GCNs) decouple sampling from forward and backward propagation during minibatch training, enhancing scalability with respect to layer depth and graph size. We propose HIS_GCNs, a hierarchical importance sampling-based learning method. By constructing minibatches using sampled subgraphs, HIS_GCNs focuses on the importance of both the core and periphery in a scale-free training graph. Specifically, it preserves the centrum of the core in most minibatches, which maintains connectivity between periphery nodes, and samples periphery edges without core node interference, which allows longer chains composed entirely of low-degree nodes remain within the same minibatch. HIS_GCNs can maximize the discrete Ricci curvature (i.e., Ollivier-Ricci curvatures) of the edges in a subgraph, enabling preservation of important chains for information propagation. This approach can achieve a low node embedding variance and a high convergence speed. Diverse experiments on Graph Neural Networks (GNNs) with node classification tasks confirmed the superior performance of HIS_GCNs in terms of both accuracy and training time. Open-source code (https://github.com/HuQiaCHN/HIS-GCN).