FieldFormer: Physics-Informed Transformers for Spatio-Temporal Field Reconstruction from Sparse Sensors
This addresses the challenge of accurate field reconstruction for applications like environmental monitoring or fluid dynamics, though it appears to be an incremental improvement over existing physics-informed learning methods.
The paper tackles the problem of reconstructing spatio-temporal fields from sparse, noisy sensor data by introducing FieldFormer, a transformer-based framework that combines data-driven learning with physics constraints. The method consistently outperforms strong baselines by more than 40% across three benchmarks, achieving accurate reconstruction with RMSE < 10^-2 from sparse (0.4%-2%) and noisy (10%) data.
Spatio-temporal sensor data is often sparse, noisy, and irregular, and existing interpolation or learning methods struggle here because they either ignore governing PDEs or do not scale. We introduce FieldFormer, a transformer-based framework for mesh-free spatio-temporal field reconstruction that combines data-driven flexibility with physics-based structure. For each query, FieldFormer gathers a local neighborhood using a learnable velocity-scaled distance metric, enabling anisotropic adaptation to different propagation regimes. Neighborhoods are built efficiently via per-batch offset recomputation, and refined in an expectation-maximization style as the velocity scales evolve. Predictions are made by a local transformer encoder, and physics consistency is enforced through autograd-based PDE residuals and boundary-specific penalties. Across three benchmarks--a scalar anisotropic heat equation, a vector-valued shallow-water system, and a realistic advection-diffusion pollution simulation--FieldFormer consistently outperforms strong baselines by more than 40%. Our results demonstrate that FieldFormer enables accurate (RMSE$<10^{-2}$), efficient, and physically consistent field reconstruction from sparse (0.4%-2%) and noisy(10%) data.