Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation
This work addresses a specific challenge in Bayesian optimization for decision-makers dealing with uncontrollable environmental variables, representing an incremental improvement by extending existing methods to handle unknown continuous context distributions.
The paper tackles the problem of optimizing expensive black-box functions affected by continuous context variables with unknown distributions, proposing two algorithms using kernel density estimation to learn the distribution online and achieving sub-linear Bayesian cumulative regret in theoretical results.
Bayesian optimization (BO) is a sample-efficient method and has been widely used for optimizing expensive black-box functions. Recently, there has been a considerable interest in BO literature in optimizing functions that are affected by context variable in the environment, which is uncontrollable by decision makers. In this paper, we focus on the optimization of functions' expectations over continuous context variable, subject to an unknown distribution. To address this problem, we propose two algorithms that employ kernel density estimation to learn the probability density function (PDF) of continuous context variable online. The first algorithm is simpler, which directly optimizes the expectation under the estimated PDF. Considering that the estimated PDF may have high estimation error when the true distribution is complicated, we further propose the second algorithm that optimizes the distributionally robust objective. Theoretical results demonstrate that both algorithms have sub-linear Bayesian cumulative regret on the expectation objective. Furthermore, we conduct numerical experiments to empirically demonstrate the effectiveness of our algorithms.