Out-of-Distribution Graph Models Merging
This addresses the challenge of model generalization for graph neural networks across heterogeneous domains, but it appears incremental as it builds on existing merging and adaptation techniques.
The paper tackles the problem of merging multiple pre-trained graph models from different domains with distribution discrepancies to construct a generalized model, achieving effectiveness as demonstrated by theoretical analysis and experimental results.
This paper studies a novel problem of out-of-distribution graph models merging, which aims to construct a generalized model from multiple graph models pre-trained on different domains with distribution discrepancy. This problem is challenging because of the difficulty in learning domain-invariant knowledge implicitly in model parameters and consolidating expertise from potentially heterogeneous GNN backbones. In this work, we propose a graph generation strategy that instantiates the mixture distribution of multiple domains. Then, we merge and fine-tune the pre-trained graph models via a MoE module and a masking mechanism for generalized adaptation. Our framework is architecture-agnostic and can operate without any source/target domain data. Both theoretical analysis and experimental results demonstrate the effectiveness of our approach in addressing the model generalization problem.