Using Distance Correlation for Efficient Bayesian Optimization
This work addresses the need for efficient optimization in science, engineering, and medicine, but it is incremental as it builds on existing Bayesian optimization methods with a new integration.
The paper tackles the problem of expensive black-box function optimization, such as hyperparameter tuning in AI, by proposing BDC, a Bayesian optimization scheme that integrates Distance Correlation to automatically balance exploration and exploitation without manual hyperparameter tuning. The result shows that BDC performs on par with popular methods like expected improvement and max-value entropy search on benchmark tests and is effective in optimizing sequential integral observations.
The need to collect data via expensive measurements of black-box functions is prevalent across science, engineering and medicine. As an example, hyperparameter tuning of a large AI model is critical to its predictive performance but is generally time-consuming and unwieldy. Bayesian optimization (BO) is a collection of methods that aim to address this issue by means of Bayesian statistical inference. In this work, we put forward a BO scheme named BDC, which integrates BO with a statistical measure of association of two random variables called Distance Correlation. BDC balances exploration and exploitation automatically, and requires no manual hyperparameter tuning. We evaluate BDC on a range of benchmark tests and observe that it performs on per with popular BO methods such as the expected improvement and max-value entropy search. We also apply BDC to optimization of sequential integral observations of an unknown terrain and confirm its utility.