Subset-of-Data Variational Inference for Deep Gaussian-Processes Regression
This work addresses training difficulties for researchers using deep Gaussian processes, though it is incremental in nature.
The paper tackled the challenge of training deep Gaussian processes by simplifying sparse approximations, reducing trainable parameters and computational cost without significant performance loss, as shown in empirical regression results.
Deep Gaussian Processes (DGPs) are multi-layer, flexible extensions of Gaussian processes but their training remains challenging. Sparse approximations simplify the training but often require optimization over a large number of inducing inputs and their locations across layers. In this paper, we simplify the training by setting the locations to a fixed subset of data and sampling the inducing inputs from a variational distribution. This reduces the trainable parameters and computation cost without significant performance degradations, as demonstrated by our empirical results on regression problems. Our modifications simplify and stabilize DGP training while making it amenable to sampling schemes for setting the inducing inputs.