Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations
This work provides a robust Bayesian extension to differential Gaussian processes, advancing their applicability in handling complex dynamic systems for real-world scenarios, though it is incremental in nature.
The authors tackled the problem of uncertainty in kernel hyperparameters in differential Gaussian processes by introducing a fully Bayesian framework that models hyperparameters as random variables and uses stochastic differential equations to learn their posterior distributions, resulting in significant improvements in flexibility and accuracy over traditional methods.
Deep Gaussian process models typically employ discrete hierarchies, but recent advancements in differential Gaussian processes (DiffGPs) have extended these models to infinite depths. However, existing DiffGP approaches often overlook the uncertainty in kernel hyperparameters by treating them as fixed and time-invariant, which degrades the model's predictive performance and neglects the posterior distribution. In this work, we introduce a fully Bayesian framework that models kernel hyperparameters as random variables and utilizes coupled stochastic differential equations (SDEs) to jointly learn their posterior distributions alongside those of inducing points. By incorporating the estimation uncertainty of hyperparameters, our method significantly enhances model flexibility and adaptability to complex dynamic systems. Furthermore, we employ a black-box adaptive SDE solver with a neural network to achieve realistic, time varying posterior approximations, thereby improving the expressiveness of the variational posterior. Comprehensive experimental evaluations demonstrate that our approach outperforms traditional methods in terms of flexibility, accuracy, and other key performance metrics. This work not only provides a robust Bayesian extension to DiffGP models but also validates its effectiveness in handling intricate dynamic behaviors, thereby advancing the applicability of Gaussian process models in diverse real-world scenarios.