Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation
This work addresses inefficiencies in parallel hyperparameter tuning for machine learning practitioners, though it appears incremental as it builds on existing batch BO methods.
The paper tackled the problem of idle workers in batch Bayesian optimization by developing an asynchronous approach called PLAyBOOK, which demonstrated improved performance over synchronous methods in both wall-clock time and number of function evaluations on synthetic and real-world tasks.
Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle. We address this problem by developing an approach, Penalising Locally for Asynchronous Bayesian Optimisation on $k$ workers (PLAyBOOK), for asynchronous parallel BO. We demonstrate empirically the efficacy of PLAyBOOK and its variants on synthetic tasks and a real-world problem. We undertake a comparison between synchronous and asynchronous BO, and show that asynchronous BO often outperforms synchronous batch BO in both wall-clock time and number of function evaluations.