On the Convergence and Size Transferability of Continuous-depth Graph Neural Networks
This work addresses the theoretical understanding of GNDEs for researchers in graph machine learning, providing incremental advances in convergence analysis and transferability bounds.
The paper tackled the problem of analyzing the convergence and size transferability of continuous-depth graph neural networks (GNDEs) by introducing Graphon-NDEs as their infinite-node limit and proving trajectory-wise convergence with explicit rates under deterministic graph sampling regimes, resulting in theoretical bounds that justify transferring models from moderate-sized to larger graphs without retraining, supported by numerical experiments.
Continuous-depth graph neural networks, also known as Graph Neural Differential Equations (GNDEs), combine the structural inductive bias of Graph Neural Networks (GNNs) with the continuous-depth architecture of Neural ODEs, offering a scalable and principled framework for modeling dynamics on graphs. In this paper, we present a rigorous convergence analysis of GNDEs with time-varying parameters in the infinite-node limit, providing theoretical insights into their size transferability. To this end, we introduce Graphon Neural Differential Equations (Graphon-NDEs) as the infinite-node limit of GNDEs and establish their well-posedness. Leveraging tools from graphon theory and dynamical systems, we prove the trajectory-wise convergence of GNDE solutions to Graphon-NDE solutions. Moreover, we derive explicit convergence rates under two deterministic graph sampling regimes: (1) weighted graphs sampled from smooth graphons, and (2) unweighted graphs sampled from $\{0,1\}$-valued (discontinuous) graphons. We further establish size transferability bounds, providing theoretical justification for the practical strategy of transferring GNDE models trained on moderate-sized graphs to larger, structurally similar graphs without retraining. Numerical experiments using synthetic and real data support our theoretical findings.