MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention
This addresses the problem of scaling neural solvers for industrial physics simulations, offering a more efficient solution for researchers and engineers in computational physics and engineering.
The paper tackles the scalability challenge in neural solvers for industrial-scale physics simulations by introducing the Multi-Scale Patch Transformer (MSPT), which efficiently captures both local interactions and global dependencies across millions of spatial elements, achieving state-of-the-art accuracy with substantially lower memory footprint and computational cost on PDE benchmarks and large-scale aerodynamic datasets.
A key scalability challenge in neural solvers for industrial-scale physics simulations is efficiently capturing both fine-grained local interactions and long-range global dependencies across millions of spatial elements. We introduce the Multi-Scale Patch Transformer (MSPT), an architecture that combines local point attention within patches with global attention to coarse patch-level representations. To partition the input domain into spatially-coherent patches, we employ ball trees, which handle irregular geometries efficiently. This dual-scale design enables MSPT to scale to millions of points on a single GPU. We validate our method on standard PDE benchmarks (elasticity, plasticity, fluid dynamics, porous flow) and large-scale aerodynamic datasets (ShapeNet-Car, Ahmed-ML), achieving state-of-the-art accuracy with substantially lower memory footprint and computational cost.