Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT
This provides a practical tool for researchers and practitioners using R to optimize deep learning hyperparameters, but it is incremental as it combines existing packages.
The paper tackles hyperparameter tuning for deep learning models by presenting a surrogate model-based approach using SPOT, demonstrating it on a neural network with MNIST data and achieving full accessibility from R with minimal code.
A surrogate model based hyperparameter tuning approach for deep learning is presented. This article demonstrates how the architecture-level parameters (hyperparameters) of deep learning models that were implemented in Keras/tensorflow can be optimized. The implementation of the tuning procedure is 100% accessible from R, the software environment for statistical computing. With a few lines of code, existing R packages (tfruns and SPOT) can be combined to perform hyperparameter tuning. An elementary hyperparameter tuning task (neural network and the MNIST data) is used to exemplify this approach