Representation Learning for Spatiotemporal Physical Systems
This work addresses the challenge of improving computational efficiency and accuracy for scientific tasks in physical systems, though it is incremental as it builds on existing self-supervised learning methods.
The paper tackles the problem of learning physics-grounded representations for spatiotemporal physical systems, focusing on downstream tasks like parameter estimation rather than next-frame prediction, and finds that latent-space methods like JEPAs outperform pixel-level prediction methods.
Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.