Predicting Ordinary Differential Equations with Transformers
This provides a scalable tool for scientists and engineers to infer governing laws from observational data, though it is incremental as it adapts transformers to a known bottleneck in ODE discovery.
The authors tackled the problem of recovering symbolic ordinary differential equations (ODEs) from noisy, irregularly sampled solution data, developing a transformer-based model that matches or outperforms existing methods in accuracy and scales efficiently after pretraining.
We develop a transformer-based sequence-to-sequence model that recovers scalar ordinary differential equations (ODEs) in symbolic form from irregularly sampled and noisy observations of a single solution trajectory. We demonstrate in extensive empirical evaluations that our model performs better or on par with existing methods in terms of accurate recovery across various settings. Moreover, our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing law of a new observed solution in a few forward passes of the model.