Graduated Optimization of Black-Box Functions
This addresses hyperparameter tuning for machine learning practitioners, but appears incremental as it builds on existing optimization methods.
The paper tackles the problem of tuning hyperparameters in machine learning by proposing a new approach for gradually and adaptively optimizing unknown functions using estimated gradients, demonstrating advantages in both low and high dimensional problems.
Motivated by the problem of tuning hyperparameters in machine learning, we present a new approach for gradually and adaptively optimizing an unknown function using estimated gradients. We validate the empirical performance of the proposed idea on both low and high dimensional problems. The experimental results demonstrate the advantages of our approach for tuning high dimensional hyperparameters in machine learning.