Information Theoretic Bayesian Optimization over the Probability Simplex
This addresses the challenge of optimizing probabilities and mixtures in domains like machine learning and robotics, representing an incremental improvement over existing methods.
The paper tackles the problem of Bayesian optimization over the probability simplex, a constrained non-Euclidean domain, by introducing α-GaBO, a novel family of algorithms grounded in information geometry. The result shows increased performance compared to constrained Euclidean approaches on benchmark functions and real-world applications like mixtures and robotic control.
Bayesian optimization is a data-efficient technique that has been shown to be extremely powerful to optimize expensive, black-box, and possibly noisy objective functions. Many applications involve optimizing probabilities and mixtures which naturally belong to the probability simplex, a constrained non-Euclidean domain defined by non-negative entries summing to one. This paper introduces $α$-GaBO, a novel family of Bayesian optimization algorithms over the probability simplex. Our approach is grounded in information geometry, a branch of Riemannian geometry which endows the simplex with a Riemannian metric and a class of connections. Based on information geometry theory, we construct Matérn kernels that reflect the geometry of the probability simplex, as well as a one-parameter family of geometric optimizers for the acquisition function. We validate our method on benchmark functions and on a variety of real-world applications including mixtures of components, mixtures of classifiers, and a robotic control task, showing its increased performance compared to constrained Euclidean approaches.