Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEs
This addresses the computational bottleneck in variational inference for latent SDEs, offering a scalable solution for researchers and practitioners dealing with time-series data.
The paper tackles the problem of inferring latent stochastic differential equations (SDEs) with high computational efficiency, achieving similar performance to existing methods while reducing model evaluations by more than an order of magnitude during training.
We consider the problem of inferring latent stochastic differential equations (SDEs) with a time and memory cost that scales independently with the amount of data, the total length of the time series, and the stiffness of the approximate differential equations. This is in stark contrast to typical methods for inferring latent differential equations which, despite their constant memory cost, have a time complexity that is heavily dependent on the stiffness of the approximate differential equation. We achieve this computational advancement by removing the need to solve differential equations when approximating gradients using a novel amortization strategy coupled with a recently derived reparametrization of expectations under linear SDEs. We show that, in practice, this allows us to achieve similar performance to methods based on adjoint sensitivities with more than an order of magnitude fewer evaluations of the model in training.