Adaptive Optimizer for Automated Hyperparameter Optimization Problem
This addresses the critical need for efficient hyperparameter tuning in machine learning, though it appears incremental as it builds on existing optimizers like Bayesian methods.
The paper tackles the problem of hyperparameter optimization by presenting a general framework for constructing adaptive optimizers that automatically adjust algorithms and parameters during optimization, demonstrating effectiveness through an example using genetic algorithm with Bayesian Optimizer and noting advantages in parallel optimization.
The choices of hyperparameters have critical effects on the performance of machine learning models. In this paper, we present a general framework that is able to construct an adaptive optimizer, which automatically adjust the appropriate algorithm and parameters in the process of optimization. Examining the method of adaptive optimizer, we product an example of using genetic algorithm to construct an adaptive optimizer based on Bayesian Optimizer and compared effectiveness with original optimizer. Especially, It has great advantages in parallel optimization.