Continual Learning on Dynamic Graphs via Parameter Isolation
This addresses the problem of continual learning on dynamic graphs for applications like social networks or recommendation systems, but it is incremental as it builds on existing parameter isolation ideas.
The paper tackles catastrophic forgetting in dynamic graph learning by proposing PI-GNN, which uses parameter isolation and expansion to learn new patterns while preserving old ones, achieving improved performance over state-of-the-art baselines on eight real-world datasets.
Many real-world graph learning tasks require handling dynamic graphs where new nodes and edges emerge. Dynamic graph learning methods commonly suffer from the catastrophic forgetting problem, where knowledge learned for previous graphs is overwritten by updates for new graphs. To alleviate the problem, continual graph learning methods are proposed. However, existing continual graph learning methods aim to learn new patterns and maintain old ones with the same set of parameters of fixed size, and thus face a fundamental tradeoff between both goals. In this paper, we propose Parameter Isolation GNN (PI-GNN) for continual learning on dynamic graphs that circumvents the tradeoff via parameter isolation and expansion. Our motivation lies in that different parameters contribute to learning different graph patterns. Based on the idea, we expand model parameters to continually learn emerging graph patterns. Meanwhile, to effectively preserve knowledge for unaffected patterns, we find parameters that correspond to them via optimization and freeze them to prevent them from being rewritten. Experiments on eight real-world datasets corroborate the effectiveness of PI-GNN compared to state-of-the-art baselines.