Doubly Bayesian Optimization
This work offers a more accessible and robust BO method for practitioners, particularly non-experts, by leveraging probabilistic programming to handle uncertainties and incorporate domain knowledge, but it appears incremental as it builds on existing BO techniques.
The paper tackles the challenge of improving Bayesian optimization (BO) by addressing issues like noisy, non-smooth, and high-dimensional domains, as well as inner-optimization, using a probabilistic program embedding. It demonstrates effectiveness on optimization benchmarks and a real-world drug development scenario, though no concrete numbers are provided in the abstract.
Probabilistic programming systems enable users to encode model structure and naturally reason about uncertainties, which can be leveraged towards improved Bayesian optimization (BO) methods. Here we present a probabilistic program embedding of BO that is capable of addressing main issues such as problematic domains (noisy, non-smooth, high-dimensional) and the neglected inner-optimization. Not only can we utilize programmable structure to incorporate domain knowledge to aid optimization, but dealing with uncertainties and implementing advanced BO techniques become trivial, crucial for use in practice (particularly for non-experts). We demonstrate the efficacy of the approach on optimization benchmarks and a real-world drug development scenario.