Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay
This addresses the problem of incremental graph structure learning for researchers in graph-based AI, though it is incremental as it adapts existing experience replay methods to GNNs.
The paper tackles catastrophic forgetting in Graph Neural Networks (GNNs) for continual learning by proposing ER-GNN, an experience replay framework with three node selection strategies, achieving effectiveness demonstrated on three benchmark datasets.
Graph Neural Networks (GNNs) have recently received significant research attention due to their superior performance on a variety of graph-related learning tasks. Most of the current works focus on either static or dynamic graph settings, addressing a single particular task, e.g., node/graph classification, link prediction. In this work, we investigate the question: can GNNs be applied to continuously learning a sequence of tasks? Towards that, we explore the Continual Graph Learning (CGL) paradigm and present the Experience Replay based framework ER-GNN for CGL to alleviate the catastrophic forgetting problem in existing GNNs. ER-GNN stores knowledge from previous tasks as experiences and replays them when learning new tasks to mitigate the catastrophic forgetting issue. We propose three experience node selection strategies: mean of feature, coverage maximization, and influence maximization, to guide the process of selecting experience nodes. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our ER-GNN and shed light on the incremental graph (non-Euclidean) structure learning.