Ripple Walk Training: A Subgraph-based training framework for Large and Deep Graph Neural Network
This addresses efficiency and scalability issues for researchers and practitioners using GNNs on large graphs, though it appears incremental as it builds on existing subgraph-based methods with a novel sampler.
The paper tackles the problems of neighbors explosion, node dependence, and oversmoothing in large or deep graph neural networks (GNNs) by proposing Ripple Walk Training (RWT), a subgraph-based training framework, which improves training efficiency and reduces space complexity, as demonstrated through extensive experiments on various graph sizes and GNN models like GCN and GAT.
Graph neural networks (GNNs) have achieved outstanding performance in learning graph-structured data and various tasks. However, many current GNNs suffer from three common problems when facing large-size graphs or using a deeper structure: neighbors explosion, node dependence, and oversmoothing. Such problems attribute to the data structures of the graph itself or the designing of the multi-layers GNNs framework, and can lead to low training efficiency and high space complexity. To deal with these problems, in this paper, we propose a general subgraph-based training framework, namely Ripple Walk Training (RWT), for deep and large graph neural networks. RWT samples subgraphs from the full graph to constitute a mini-batch, and the full GNN is updated based on the mini-batch gradient. We analyze the high-quality subgraphs to train GNNs in a theoretical way. A novel sampling method Ripple Walk Sampler works for sampling these high-quality subgraphs to constitute the mini-batch, which considers both the randomness and connectivity of the graph-structured data. Extensive experiments on different sizes of graphs demonstrate the effectiveness and efficiency of RWT in training various GNNs (GCN & GAT).