Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
For researchers and practitioners in spatio-temporal modeling, NeoST offers a scalable solution to distributional bias and structural bottlenecks, achieving SOTA performance without real-world pre-training data.
NeoST is the first spatio-temporal foundation model pre-trained solely on synthetic data, outperforming existing STFMs across diverse real-world benchmarks with superior long-horizon stability and inference efficiency.
Spatio-Temporal Foundation Models (STFMs) aim to learn generalizable representations of complex dynamical systems across space and time. However, existing approaches suffer from distributional bias in real-world pre-training data, structural bottlenecks of autoregressive or diffusion-based paradigms, and objectives that overemphasize point-wise reconstruction in noisy observation space.We propose \textbf{NeoST}, the first spatio-temporal foundation model pre-trained solely on procedurally generated synthetic systems. NeoST introduces a scalable synthetic pre-training corpus to mitigate real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-world spatio-temporal systems, achieves superior long-horizon stability and inference efficiency.