Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
This work addresses the challenge of principled fusion of Gaussian process predictions for scalable and robust machine learning applications, representing an incremental improvement over prior combination schemes.
The authors tackled the problem of combining predictions from multiple probabilistic models by proposing a generalized product of experts (gPoE) framework, showing that it satisfies four key properties for scalability, expressiveness, robustness, and validity, unlike existing methods.
In this work, we propose a generalized product of experts (gPoE) framework for combining the predictions of multiple probabilistic models. We identify four desirable properties that are important for scalability, expressiveness and robustness, when learning and inferring with a combination of multiple models. Through analysis and experiments, we show that gPoE of Gaussian processes (GP) have these qualities, while no other existing combination schemes satisfy all of them at the same time. The resulting GP-gPoE is highly scalable as individual GP experts can be independently learned in parallel; very expressive as the way experts are combined depends on the input rather than fixed; the combined prediction is still a valid probabilistic model with natural interpretation; and finally robust to unreliable predictions from individual experts.