Deep Transformed Gaussian Processes
This work addresses the need for more flexible stochastic process models in machine learning, though it appears incremental as it builds on existing Transformed and Deep Gaussian Processes.
The authors tackled the problem of increasing the flexibility of Gaussian Processes by proposing Deep Transformed Gaussian Processes (DTGPs), a multi-layer model that combines transformed and hierarchical layers, achieving competitive performance and scalability in regression tasks.
Transformed Gaussian Processes (TGPs) are stochastic processes specified by transforming samples from the joint distribution from a prior process (typically a GP) using an invertible transformation; increasing the flexibility of the base process. Furthermore, they achieve competitive results compared with Deep Gaussian Processes (DGPs), which are another generalization constructed by a hierarchical concatenation of GPs. In this work, we propose a generalization of TGPs named Deep Transformed Gaussian Processes (DTGPs), which follows the trend of concatenating layers of stochastic processes. More precisely, we obtain a multi-layer model in which each layer is a TGP. This generalization implies an increment of flexibility with respect to both TGPs and DGPs. Exact inference in such a model is intractable. However, we show that one can use variational inference to approximate the required computations yielding a straightforward extension of the popular DSVI inference algorithm Salimbeni et al (2017). The experiments conducted evaluate the proposed novel DTGPs in multiple regression datasets, achieving good scalability and performance.