Bayesian Optimization for Iterative Learning
This work addresses the costly hyperparameter tuning process for deep learning practitioners, offering an incremental improvement over traditional methods by leveraging iterative training data.
The paper tackles the expensive problem of hyperparameter tuning in deep learning by proposing a Bayesian optimization approach that uses intermediate training information to compress learning progress into a single score, balancing assessment benefits against computation costs. It demonstrates efficiency by outperforming existing baselines in identifying optimal hyperparameters for deep reinforcement learning and convolutional neural networks in minimal time.
The performance of deep (reinforcement) learning systems crucially depends on the choice of hyperparameters. Their tuning is notoriously expensive, typically requiring an iterative training process to run for numerous steps to convergence. Traditional tuning algorithms only consider the final performance of hyperparameters acquired after many expensive iterations and ignore intermediate information from earlier training steps. In this paper, we present a Bayesian optimization (BO) approach which exploits the iterative structure of learning algorithms for efficient hyperparameter tuning. We propose to learn an evaluation function compressing learning progress at any stage of the training process into a single numeric score according to both training success and stability. Our BO framework is then balancing the benefit of assessing a hyperparameter setting over additional training steps against their computation cost. We further increase model efficiency by selectively including scores from different training steps for any evaluated hyperparameter set. We demonstrate the efficiency of our algorithm by tuning hyperparameters for the training of deep reinforcement learning agents and convolutional neural networks. Our algorithm outperforms all existing baselines in identifying optimal hyperparameters in minimal time.