Comparison of neural closure models for discretised PDEs
This work addresses the challenge of improving stability and accuracy in neural closure models for computational fluid dynamics and related fields, representing an incremental advancement by systematically comparing existing methods.
The paper tackled the problem of selecting optimal training procedures for neural closure models in multiscale PDE systems, finding that trajectory fitting with discretise-then-optimise yields the most accurate models, outperforming derivative fitting and optimise-then-discretise approaches on test problems like Kuramoto-Sivashinsky and Burgers equations.
Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term.