Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties
This work addresses the problem of enhancing predictive uncertainty calibration in probabilistic deep learning models for researchers and practitioners in machine learning, representing an incremental improvement over existing methods.
The authors tackled the challenge of improving predictive uncertainties in Deep Gaussian Processes by introducing a novel Gaussian variational family that retains covariances between latent processes while enabling fast convergence through marginalization of global latent variables. Their method achieved excellent results on benchmark datasets, striking a better balance between accuracy and calibrated uncertainty estimates compared to state-of-the-art alternatives.
Deep Gaussian Processes learn probabilistic data representations for supervised learning by cascading multiple Gaussian Processes. While this model family promises flexible predictive distributions, exact inference is not tractable. Approximate inference techniques trade off the ability to closely resemble the posterior distribution against speed of convergence and computational efficiency. We propose a novel Gaussian variational family that allows for retaining covariances between latent processes while achieving fast convergence by marginalising out all global latent variables. After providing a proof of how this marginalisation can be done for general covariances, we restrict them to the ones we empirically found to be most important in order to also achieve computational efficiency. We provide an efficient implementation of our new approach and apply it to several benchmark datasets. It yields excellent results and strikes a better balance between accuracy and calibrated uncertainty estimates than its state-of-the-art alternatives.