Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning
This work addresses the challenge of robust uncertainty quantification in nonlinear dynamical systems for domains like time series analysis, though it appears incremental as it builds on existing physics-informed and Gaussian process frameworks.
The authors tackled modeling highly nonlinear dynamical systems with uncertainty quantification by introducing the deep latent force model (DLFM), a deep Gaussian process with physics-informed kernels derived from ODEs, and demonstrated its capability to capture dynamics in real-world multi-output time series data while achieving comparable performance to non-physics-informed models on benchmark tasks.
Modelling the behaviour of highly nonlinear dynamical systems with robust uncertainty quantification is a challenging task which typically requires approaches specifically designed to address the problem at hand. We introduce a domain-agnostic model to address this issue termed the deep latent force model (DLFM), a deep Gaussian process with physics-informed kernels at each layer, derived from ordinary differential equations using the framework of process convolutions. Two distinct formulations of the DLFM are presented which utilise weight-space and variational inducing points-based Gaussian process approximations, both of which are amenable to doubly stochastic variational inference. We present empirical evidence of the capability of the DLFM to capture the dynamics present in highly nonlinear real-world multi-output time series data. Additionally, we find that the DLFM is capable of achieving comparable performance to a range of non-physics-informed probabilistic models on benchmark univariate regression tasks. We also empirically assess the negative impact of the inducing points framework on the extrapolation capabilities of LFM-based models.