BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters
This provides a practical solution for researchers and practitioners needing to reduce computational expenses in hyperparameter tuning, though it is incremental as it builds on existing methods like Bayesian optimization and HyperBand.
The authors tackled the high cost of hyperparameter optimization and neural architecture search by introducing a Python tool suite that combines Bayesian optimization with HyperBand for multi-fidelity optimization, enabling efficient and robust analysis of optimization runs.
Hyperparameter optimization and neural architecture search can become prohibitively expensive for regular black-box Bayesian optimization because the training and evaluation of a single model can easily take several hours. To overcome this, we introduce a comprehensive tool suite for effective multi-fidelity Bayesian optimization and the analysis of its runs. The suite, written in Python, provides a simple way to specify complex design spaces, a robust and efficient combination of Bayesian optimization and HyperBand, and a comprehensive analysis of the optimization process and its outcomes.