Gaussian Process Conditional Density Estimation
This work addresses conditional density estimation for applications like spatio-temporal modeling and few-shot learning, representing an incremental improvement with a novel hybrid method.
The authors tackled the problem of conditional density estimation by extending model inputs with latent variables and using Gaussian processes to map to conditional distribution samples, enabling modeling of small datasets and application to big data via stochastic variational inference.
Conditional Density Estimation (CDE) models deal with estimating conditional distributions. The conditions imposed on the distribution are the inputs of the model. CDE is a challenging task as there is a fundamental trade-off between model complexity, representational capacity and overfitting. In this work, we propose to extend the model's input with latent variables and use Gaussian processes (GP) to map this augmented input onto samples from the conditional distribution. Our Bayesian approach allows for the modeling of small datasets, but we also provide the machinery for it to be applied to big data using stochastic variational inference. Our approach can be used to model densities even in sparse data regions, and allows for sharing learned structure between conditions. We illustrate the effectiveness and wide-reaching applicability of our model on a variety of real-world problems, such as spatio-temporal density estimation of taxi drop-offs, non-Gaussian noise modeling, and few-shot learning on omniglot images.