Training-Free Cross-Architecture Merging for Graph Neural Networks
This addresses a bottleneck for practitioners needing to combine specialized GNN models efficiently, though it is incremental as it extends model merging to a specific domain.
The paper tackles the problem of merging graph neural networks (GNNs) with heterogeneous architectures, which is unreliable with existing methods due to topology-dependent message passing, by introducing H-GRAMA, a training-free framework that lifts merging to operator space, achieving inference speedups of 1.2x to 1.9x over ensembles while retaining high specialist accuracy in most cases.
Model merging has emerged as a powerful paradigm for combining the capabilities of distinct expert models without the high computational cost of retraining, yet current methods are fundamentally constrained to homogeneous architectures. For GNNs, however, message passing is topology-dependent and sensitive to misalignment, making direct parameter-space merging unreliable. To bridge this gap, we introduce H-GRAMA (Heterogeneous Graph Routing and Message Alignment), a training-free framework that lifts merging from parameter space to operator space. We formalize Universal Message Passing Mixture (UMPM), a shared operator family that expresses heterogeneous GNN layers in a common functional language. H-GRAMA enables cross-architecture GNN merging (e.g., GCN to GAT) without retraining, retaining high specialist accuracy in most cases in compatible depth settings and achieving inference speedups of 1.2x to 1.9x over ensembles.