Neighbor2Seq: Deep Learning on Massive Graphs by Transforming Neighbors to Sequences
This addresses the computational and memory bottlenecks for researchers and practitioners applying GNNs to large-scale real-world graph data, offering a scalable solution.
The paper tackles the scalability problem of graph neural networks (GNNs) on massive graphs by proposing Neighbor2Seq, which transforms node neighborhoods into sequences to enable efficient mini-batch training with deep learning operations like convolution and attention, achieving superior performance on graphs with over 111 million nodes and 1.6 billion edges.
Modern graph neural networks (GNNs) use a message passing scheme and have achieved great success in many fields. However, this recursive design inherently leads to excessive computation and memory requirements, making it not applicable to massive real-world graphs. In this work, we propose the Neighbor2Seq to transform the hierarchical neighborhood of each node into a sequence. This novel transformation enables the subsequent mini-batch training for general deep learning operations, such as convolution and attention, that are designed for grid-like data and are shown to be powerful in various domains. Therefore, our Neighbor2Seq naturally endows GNNs with the efficiency and advantages of deep learning operations on grid-like data by precomputing the Neighbor2Seq transformations. We evaluate our method on a massive graph, with more than 111 million nodes and 1.6 billion edges, as well as several medium-scale graphs. Results show that our proposed method is scalable to massive graphs and achieves superior performance across massive and medium-scale graphs. Our code is available at https://github.com/divelab/Neighbor2Seq.