Can Graph Neural Networks Go "Online"? An Analysis of Pretraining and Inference
This addresses the challenge of online learning for graph neural networks in dynamic real-world applications, but it is incremental as it focuses on a specific experimental setup.
The paper tackled the problem of adapting graph neural networks to dynamic graphs with new nodes and edges after training, finding that pretrained models achieve high accuracy and outperform retraining from scratch.
Large-scale graph data in real-world applications is often not static but dynamic, i. e., new nodes and edges appear over time. Current graph convolution approaches are promising, especially, when all the graph's nodes and edges are available during training. When unseen nodes and edges are inserted after training, it is not yet evaluated whether up-training or re-training from scratch is preferable. We construct an experimental setup, in which we insert previously unseen nodes and edges after training and conduct a limited amount of inference epochs. In this setup, we compare adapting pretrained graph neural networks against retraining from scratch. Our results show that pretrained models yield high accuracy scores on the unseen nodes and that pretraining is preferable over retraining from scratch. Our experiments represent a first step to evaluate and develop truly online variants of graph neural networks.