Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization tasks
This work addresses the need for a unified approach to various specialized Bayesian Optimization tasks, which is incremental as it builds on existing methods but offers a novel integration framework.
The paper tackles the problem of unifying exotic Bayesian Optimization tasks like collaborative/federated, batch, and multi-fidelity BO under a common framework using the weighted Wasserstein Barycenter of Gaussian Processes, demonstrating that each task only requires a specific weighting scheme and enabling more computationally efficient computations compared to state-of-the-art methods.
Exploiting the analogy between Gaussian Distributions and Gaussian Processes' posterior, we present how the weighted Wasserstein Barycenter of Gaussian Processes (W2BGP) can be used to unify, under a common framework, different exotic Bayesian Optimization (BO) tasks. Specifically, collaborative/federated BO, (synchronous) batch BO, and multi-fidelity BO are considered in this paper. Our empirical analysis proves that each one of these tasks requires just an appropriate weighting schema for the W2BGP, while the entire framework remains untouched. Moreover, we demonstrate that the most well-known BO acquisition functions can be easily re-interpreted under the proposed framework and also enable a more computationally efficient way to deal with the computation of the Wasserstein Barycenter, compared with state-of-the-art methods from the Machine Learning literature. Finally, research perspectives branching from the proposed approach are presented.