Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
This work addresses the meta-optimization challenges in learning-to-learn for optimization, offering theoretical insights but is incremental as it focuses on a simplified quadratic setting.
The paper tackles the problem of tuning optimization algorithm parameters via learning-to-learn, showing that naive meta-objectives cause gradient explosion/vanishing, and provides guarantees for step size tuning on quadratic loss, with empirical validation on neural network optimizers.
Choosing the right parameters for optimization algorithms is often the key to their success in practice. Solving this problem using a learning-to-learn approach -- using meta-gradient descent on a meta-objective based on the trajectory that the optimizer generates -- was recently shown to be effective. However, the meta-optimization problem is difficult. In particular, the meta-gradient can often explode/vanish, and the learned optimizer may not have good generalization performance if the meta-objective is not chosen carefully. In this paper we give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Our results show that the naïve objective suffers from meta-gradient explosion/vanishing problem. Although there is a way to design the meta-objective so that the meta-gradient remains polynomially bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the generalization performance of the learned optimizer. Finally, we verify our results empirically and show that a similar phenomenon appears even for more complicated learned optimizers parametrized by neural networks.