Rethinking Trust Region Bayesian Optimization in High Dimensions
For practitioners of high-dimensional Bayesian optimization, this work addresses a known bottleneck in TuRBO's performance with a simple fix.
The paper identifies that TuRBO's local GP model degenerates in high dimensions due to inappropriate lengthscale design, and proposes AdaScale-TuRBO which scales lengthscale with dimension and trust region size, achieving robust improvements over standard TuRBO and other methods on synthetic and real-world tasks.
Trust Region Bayesian Optimization (TuRBO) is an effective strategy for alleviating the curse of dimensionality in high-dimensional black-box optimization. However, inappropriate lengthscale design can cause the local Gaussian process (GP) model within the trust region to degenerate, leading to suboptimal performance in high dimensions. In this work, we show that TuRBO's local GP may remain either excessively complex or overly simple as the dimension $D$ and trust region side length $L$ vary. To address this issue, we propose a straightforward variant, AdaScale-TuRBO, which scales the GP lengthscale with both the problem dimension and trust region size, thereby preserving kernel geometry and maintaining consistent prior complexity. Empirically, we show that AdaScale-TuRBO can robustly outperform standard TuRBO and other popular high-dimensional BO methods on synthetic benchmarks and real-world trajectory planning tasks.