A Probabilistic State Space Model for Joint Inference from Differential Equations and Data
This work addresses the problem of efficient inference in mechanistic models for scientific machine learning applications, offering a novel integration that reduces computational demands, though it is incremental in building on probabilistic numerics and latent force models.
The paper tackles the computational challenge of combining differential equation solvers with standard inference techniques by introducing a probabilistic state space model that integrates Bayesian filtering with latent force models, enabling joint inference on latent forces and ODE solutions in a single linear complexity pass. It demonstrates the method's effectiveness by training a non-parametric SIRD model on COVID-19 outbreak data, achieving inference at the cost of computing a single ODE solution.
Mechanistic models with differential equations are a key component of scientific applications of machine learning. Inference in such models is usually computationally demanding, because it involves repeatedly solving the differential equation. The main problem here is that the numerical solver is hard to combine with standard inference techniques. Recent work in probabilistic numerics has developed a new class of solvers for ordinary differential equations (ODEs) that phrase the solution process directly in terms of Bayesian filtering. We here show that this allows such methods to be combined very directly, with conceptual and numerical ease, with latent force models in the ODE itself. It then becomes possible to perform approximate Bayesian inference on the latent force as well as the ODE solution in a single, linear complexity pass of an extended Kalman filter / smoother - that is, at the cost of computing a single ODE solution. We demonstrate the expressiveness and performance of the algorithm by training, among others, a non-parametric SIRD model on data from the COVID-19 outbreak.