PACSBO: Probably approximately correct safe Bayesian optimization
This work addresses a key bottleneck in safe optimization for control policies, offering a more practical approach with theoretical guarantees, though it is incremental in refining existing methods.
The authors tackled the problem of safe Bayesian optimization requiring an impractical smoothness assumption by proposing PACSBO, which estimates the RKHS norm from data and treats it locally, resulting in reduced conservatism and demonstrated benefits over existing algorithms in numerical and hardware experiments.
Safe Bayesian optimization (BO) algorithms promise to find optimal control policies without knowing the system dynamics while at the same time guaranteeing safety with high probability. In exchange for those guarantees, popular algorithms require a smoothness assumption: a known upper bound on a norm in a reproducing kernel Hilbert space (RKHS). The RKHS is a potentially infinite-dimensional space, and it is unclear how to, in practice, obtain an upper bound of an unknown function in its corresponding RKHS. In response, we propose an algorithm that estimates an upper bound on the RKHS norm of an unknown function from data and investigate its theoretical properties. Moreover, akin to Lipschitz-based methods, we treat the RKHS norm as a local rather than a global object, and thus reduce conservatism. Integrating the RKHS norm estimation and the local interpretation of the RKHS norm into a safe BO algorithm yields PACSBO, an algorithm for probably approximately correct safe Bayesian optimization, for which we provide numerical and hardware experiments that demonstrate its applicability and benefits over popular safe BO algorithms.