Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes
This work addresses a practical bottleneck in preferential Bayesian optimization for applications relying on pairwise comparisons, offering an incremental improvement over existing methods.
The paper tackled the challenge of preferential Bayesian optimization with computationally intractable skew Gaussian processes, revealing inaccuracies in Gaussian approximations and developing a new method that achieves high computational efficiency and low sample complexity, as demonstrated through extensive numerical experiments.
We study preferential Bayesian optimization (BO) where reliable feedback is limited to pairwise comparison called duels. An important challenge in preferential BO, which uses the preferential Gaussian process (GP) model to represent flexible preference structure, is that the posterior distribution is a computationally intractable skew GP. The most widely used approach for preferential BO is Gaussian approximation, which ignores the skewness of the true posterior. Alternatively, Markov chain Monte Carlo (MCMC) based preferential BO is also proposed. In this work, we first verify the accuracy of Gaussian approximation, from which we reveal the critical problem that the predictive probability of duels can be inaccurate. This observation motivates us to improve the MCMC-based estimation for skew GP, for which we show the practical efficiency of Gibbs sampling and derive the low variance MC estimator. However, the computational time of MCMC can still be a bottleneck in practice. Towards building a more practical preferential BO, we develop a new method that achieves both high computational efficiency and low sample complexity, and then demonstrate its effectiveness through extensive numerical experiments.