Convolutional Normalizing Flows for Deep Gaussian Processes
This work addresses the inference problem in DGPs for machine learning practitioners, offering a more efficient and accurate method, though it is incremental as it builds on existing variational inference techniques.
The paper tackles the challenge of performing exact inference in deep Gaussian processes (DGPs) by introducing a new approach using normalizing flows to specify flexible and scalable approximate posterior distributions, with empirical results showing that their convolutional normalizing flow (CNF) DGP outperforms state-of-the-art approximation methods.
Deep Gaussian processes (DGPs), a hierarchical composition of GP models, have successfully boosted the expressive power of their single-layer counterpart. However, it is impossible to perform exact inference in DGPs, which has motivated the recent development of variational inference-based methods. Unfortunately, either these methods yield a biased posterior belief or it is difficult to evaluate their convergence. This paper introduces a new approach for specifying flexible, arbitrarily complex, and scalable approximate posterior distributions. The posterior distribution is constructed through a normalizing flow (NF) which transforms a simple initial probability into a more complex one through a sequence of invertible transformations. Moreover, a novel convolutional normalizing flow (CNF) is developed to improve the time efficiency and capture dependency between layers. Empirical evaluation shows that CNF DGP outperforms the state-of-the-art approximation methods for DGPs.