RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels
This work addresses efficient black-box optimization for applications like industrial design and scientific computing, representing an incremental improvement over existing meta-learning and few-shot techniques.
The paper tackles the problem of high-dimensional Bayesian optimization by proposing RTDK-BO, which combines transformer deep kernels with reinforcement learning to improve surrogate modeling and exploration, achieving state-of-the-art results in continuous high-dimensional optimization tasks.
Bayesian Optimization (BO), guided by Gaussian process (GP) surrogates, has proven to be an invaluable technique for efficient, high-dimensional, black-box optimization, a critical problem inherent to many applications such as industrial design and scientific computing. Recent contributions have introduced reinforcement learning (RL) to improve the optimization performance on both single function optimization and \textit{few-shot} multi-objective optimization. However, even few-shot techniques fail to exploit similarities shared between closely related objectives. In this paper, we combine recent developments in Deep Kernel Learning (DKL) and attention-based Transformer models to improve the modeling powers of GP surrogates with meta-learning. We propose a novel method for improving meta-learning BO surrogates by incorporating attention mechanisms into DKL, empowering the surrogates to adapt to contextual information gathered during the BO process. We combine this Transformer Deep Kernel with a learned acquisition function trained with continuous Soft Actor-Critic Reinforcement Learning to aid in exploration. This Reinforced Transformer Deep Kernel (RTDK-BO) approach yields state-of-the-art results in continuous high-dimensional optimization problems.