Fusion of Graph Neural Networks via Optimal Transport
This work addresses model fusion for graph neural networks, but it is incremental as it builds on existing optimal transport methods without achieving broad SOTA gains.
The paper tackles the problem of combining Graph Neural Networks (GCNs) into a single model by aligning their weights layer-wise using optimal transport, and it shows that this fusion method consistently outperforms vanilla averaging, though it is harder for GCNs than MLPs and incorporating graph structure does not improve performance.
In this paper, we explore the idea of combining GCNs into one model. To that end, we align the weights of different models layer-wise using optimal transport (OT). We present and evaluate three types of transportation costs and show that the studied fusion method consistently outperforms the performance of vanilla averaging. Finally, we present results suggesting that model fusion using OT is harder in the case of GCNs than MLPs and that incorporating the graph structure into the process does not improve the performance of the method.