The observer-assisted method for adjusting hyper-parameters in deep learning algorithms
This is an incremental method for automating hyperparameter optimization in deep learning, potentially benefiting researchers and practitioners.
The paper tackles hyperparameter tuning in deep learning by introducing an external observer that models algorithm performance through random experiments to predict quality scores, enabling iterative improvement towards optimal settings.
This paper presents a concept of a novel method for adjusting hyper-parameters in Deep Learning (DL) algorithms. An external agent-observer monitors a performance of a selected Deep Learning algorithm. The observer learns to model the DL algorithm using a series of random experiments. Consequently, it may be used for predicting a response of the DL algorithm in terms of a selected quality measurement to a set of hyper-parameters. This allows to construct an ensemble composed of a series of evaluators which constitute an observer-assisted architecture. The architecture may be used to gradually iterate towards to the best achievable quality score in tiny steps governed by a unit of progress. The algorithm is stopped when the maximum number of steps is reached or no further progress is made.