Random Feature Expansions for Deep Gaussian Processes
This work addresses the problem of limited scalability and cumbersome inference for DGPs, enabling more practical deep probabilistic nonparametric learning with uncertainty quantification, though it is incremental as it builds on existing DGP methods.
The authors tackled the scalability and construction difficulties of Deep Gaussian Processes (DGPs) by introducing a novel formulation based on random feature expansions, trained with stochastic variational inference, achieving state-of-the-art inference on datasets with up to 8 million observations and architectures with up to 30 hidden layers.
The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work, we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.