On the under-reaching phenomenon in message-passing neural PDE solvers: revisiting the CFL condition
This addresses a bottleneck in neural PDE solvers for researchers and practitioners, providing theoretical guidance to improve efficiency and accuracy, though it is incremental as it builds on existing GNN methods.
The paper tackled the problem of determining the minimum number of message-passing iterations needed in graph neural networks for solving PDEs, deriving sharp lower bounds that reduce hyperparameter tuning and ensure accurate solutions when satisfied.
This paper proposes sharp lower bounds for the number of message passing iterations required in graph neural networks (GNNs) when solving partial differential equations (PDE). This significantly reduces the need for exhaustive hyperparameter tuning. Bounds are derived for the three fundamental classes of PDEs (hyperbolic, parabolic and elliptic) by relating the physical characteristics of the problem in question to the message-passing requirement of GNNs. In particular, we investigate the relationship between the physical constants of the equations governing the problem, the spatial and temporal discretisation and the message passing mechanisms in GNNs. When the number of message passing iterations is below these proposed limits, information does not propagate efficiently through the network, resulting in poor solutions, even for deep GNN architectures. In contrast, when the suggested lower bound is satisfied, the GNN parameterisation allows the model to accurately capture the underlying phenomenology, resulting in solvers of adequate accuracy. Examples are provided for four different examples of equations that show the sharpness of the proposed lower bounds.