Predicting Physics in Mesh-reduced Space with Temporal Attention
This addresses the challenge of long-term prediction in high-dimensional physics simulations for researchers and engineers, though it is incremental as it builds on existing graph-based methods.
The paper tackles the problem of error accumulation and drift in graph-based models for predicting complex physical systems on irregular meshes by proposing a transformer-style temporal attention model with an encoder-decoder structure to capture long-term dependencies in a low-dimensional representation. It outperforms a GNN baseline on fluid dynamics tasks, achieving stable rollouts without training noise and perfectly phase-stable predictions for long sequences.
Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.