When do neural ordinary differential equations generalize on complex networks?
This addresses the problem of understanding neural ODEs' behavior on complex networks for researchers in machine learning and complex systems, but it is incremental as it builds on existing neural ODE and graph theory frameworks.
The study investigated neural ODEs' generalization on graph-structured data, finding that degree heterogeneity and the type of dynamical system primarily affect their ability to generalize across graph sizes and properties, with average clustering playing a secondary role.
Neural ordinary differential equations (neural ODEs) can effectively learn dynamical systems from time series data, but their behavior on graph-structured data remains poorly understood, especially when applied to graphs with different size or structure than encountered during training. We study neural ODEs ($\mathtt{nODE}$s) with vector fields following the Barabási-Barzel form, trained on synthetic data from five common dynamical systems on graphs. Using the $\mathbb{S}^1$-model to generate graphs with realistic and tunable structure, we find that degree heterogeneity and the type of dynamical system are the primary factors in determining $\mathtt{nODE}$s' ability to generalize across graph sizes and properties. This extends to $\mathtt{nODE}$s' ability to capture fixed points and maintain performance amid missing data. Average clustering plays a secondary role in determining $\mathtt{nODE}$ performance. Our findings highlight $\mathtt{nODE}$s as a powerful approach to understanding complex systems but underscore challenges emerging from degree heterogeneity and clustering in realistic graphs.