Variational Elliptical Processes
This work addresses the need for more flexible probabilistic models in machine learning, particularly for scenarios with non-Gaussian likelihoods or heavy-tailed data, representing a novel generalization rather than an incremental improvement.
The paper tackled the problem of extending Gaussian processes to handle heavy-tailed data by introducing elliptical processes, a family of non-parametric probabilistic models that subsume Gaussian and Student's t processes, achieving computational tractability and applicability to large-scale problems through variational inference.
We present elliptical processes, a family of non-parametric probabilistic models that subsume Gaussian processes and Student's t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.