A Deterministic Approximation to Neural SDEs
This work addresses uncertainty calibration for practitioners using NSDEs, offering a more efficient alternative to Monte Carlo methods, though it is incremental as it builds on existing NSDE frameworks.
The paper tackles the problem of computationally prohibitive uncertainty quantification in Neural Stochastic Differential Equations (NSDEs) by developing a deterministic approximation method, which achieves comparable calibration quality to Monte Carlo sampling at lower cost and improves prediction accuracy in experiments.
Neural Stochastic Differential Equations (NSDEs) model the drift and diffusion functions of a stochastic process as neural networks. While NSDEs are known to make accurate predictions, their uncertainty quantification properties have been remained unexplored so far. We report the empirical finding that obtaining well-calibrated uncertainty estimations from NSDEs is computationally prohibitive. As a remedy, we develop a computationally affordable deterministic scheme which accurately approximates the transition kernel, when dynamics is governed by a NSDE. Our method introduces a bidimensional moment matching algorithm: vertical along the neural net layers and horizontal along the time direction, which benefits from an original combination of effective approximations. Our deterministic approximation of the transition kernel is applicable to both training and prediction. We observe in multiple experiments that the uncertainty calibration quality of our method can be matched by Monte Carlo sampling only after introducing high computational cost. Thanks to the numerical stability of deterministic training, our method also improves prediction accuracy.